Group 19: Phase 2 - Cats vs Dogs Detector (CaDoD)

Team Members

We are a group of 4 members:

Aishwarya Sinhasane - avsinhas@iu.edu (In picture, Left top)

Himanshu Joshi - hsjoshi@iu.edu (In picture, Right bottom)

Sreelaxmi Chakkadath - schakkad@iu.edu (In picture, Left bottom)

Sumitha Vellinalur Thattai - svtranga@iu.edu (In picture, Right top)

Project Abstract

The objective of our project is to classify images as either dogs or cats. Additionally, we also plan to find where the cat/dog is in the image. Although the task is simple to human eyes, computers find it hard to distinguish between images because of a plethora of factors including cluttered background, illumination conditions, deformations, occlusions among several others. We plan to build an end-to-end machine learning model which will help computers differentiate between cat and dog images with better accuracy.

In our previous phase, we finalized using Gradient boosting and linear regression as our baseline for image detection and boundary box detection respectively. In this phase, we extended the baseline and implemented a complex loss function (CXE + MSE) using homegrown linear and logistic regression.

Furthermore, we built multi-layered perceptron and calculated the accuracy and loss per epoch for classification and regression tasks. We have used data augmentation, dropout layers and regularization to overcome overfitting. In addition to this, we built a multi-headed predictor that calculates the combined loss function from classification and the regression tasks and uses it to optimize the weights and bias of the network.

Accuracy and loss are our primary evaluation parameters. In addition to this, we also used classification report (F1 score) and confusion matrix to evaluate the classification model. Using the above model, we got an accuracy of ~60% for classification and MSE close to zero for regression.

Project Meta Data

We have completed the following task this week:

• Homegrown linear regression

• Complex loss function (CXE + MSE) using homegrown models

  • Built sequential neural network for image classification and used the following to improve accuracy

    • Data Augmentation

    • Dropout layers

    • Regularization

• Built sequential neural network for boundary detection

• Compare the performance of the above models using Accuracy and loss
• Built a multi-headed cat-dog detector using the OOP API that combines loss function: CXE + MSE
• Implemented convolutional neural network for image classification and box detection

Data Description

The data we plan to use is the Kaggle data set. We will be using two files – one for image and the other for boundary:

The images are taken from the cadod.tar.gz

The boundary information of the images is from cadod.csv file

• There are about ~13k images in the data set. There is a good balance of classes with ~6.8K cat images and ~6.1K dog images in the data set.

Image information (cadod.tar.gz):

• There are ~13K images of various sizes and aspect ratios. Majority of the images are of 512 X 384 size

• All the images are in RGB scale

• The boundaries of the images are stored in the cadod.csv file

Attributes of the Boundary File (cadod.csv):

• This has information about the image box coordinates

• There are about 20 features in this data set

-15 numerical features

- This includes the image ID, the coordinates of the boundary boxes, and also the normalized coordinates of the boundary boxes 

-5 categorical features

-This gives information about the occlusion, depiction, truncation, etc. 

Task

The following are the end-to-end task to achieve the results:

• Image pre-processing (Augmentation)

  • Augmentation involved flipping and rotation

• Use sequential models to build neural networks

• Neural network classifier for image detection

• Neural network regressor for box detection

• Implemented the above using OOP API and created a combined loss function (CXE + MSE)

• Build a convolutional neural network

Since the data set is very large, we have carried out the above tasks were carried out only in a subset of data. Hence the results will be only directional and not accurate

from collections import Counter
import glob
import matplotlib.image as mpimg
import matplotlib.pyplot as plt
import numpy as np
import os
import pandas as pd
from PIL import Image
from sklearn.exceptions import ConvergenceWarning
from sklearn.linear_model import SGDClassifier, SGDRegressor
from sklearn.metrics import accuracy_score, mean_squared_error, roc_auc_score
from sklearn.model_selection import train_test_split
import tarfile
from tqdm import tqdm
import warnings
import seaborn as sns
from sklearn.model_selection import ShuffleSplit
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
from sklearn.metrics import f1_score
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier,AdaBoostClassifier,GradientBoostingClassifier
from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import randint

Import Data

Unarchive data

def extract_tar(file, path):
    """
    function to extract tar.gz files to specified location
    
    Args:
        file (str): path where the file is located
        path (str): path where you want to extract
    """
    with tarfile.open(file) as tar:
        files_extracted = 0
        for member in tqdm(tar.getmembers()):
            if os.path.isfile(path + member.name[1:]):
                continue
            else:
                tar.extract(member, path)
                files_extracted += 1
        tar.close()
        if files_extracted < 3:
            print('Files already exist')

path = 'images/'

# extract_tar('cadod.tar.gz', path)

Load bounding box meta data

df = pd.read_csv('cadod.csv')

df.head()

	ImageID	Source	LabelName	Confidence	XMin	XMax	YMin	YMax	IsOccluded	IsTruncated	...	IsDepiction	IsInside	XClick1X	XClick2X	XClick3X	XClick4X	XClick1Y	XClick2Y	XClick3Y	XClick4Y
0	0000b9fcba019d36	xclick	/m/0bt9lr	1	0.165000	0.903750	0.268333	0.998333	1	1	...	0	0	0.636250	0.903750	0.748750	0.165000	0.268333	0.506667	0.998333	0.661667
1	0000cb13febe0138	xclick	/m/0bt9lr	1	0.000000	0.651875	0.000000	0.999062	1	1	...	0	0	0.312500	0.000000	0.317500	0.651875	0.000000	0.410882	0.999062	0.999062
2	0005a9520eb22c19	xclick	/m/0bt9lr	1	0.094167	0.611667	0.055626	0.998736	1	1	...	0	0	0.487500	0.611667	0.243333	0.094167	0.055626	0.226296	0.998736	0.305942
3	0006303f02219b07	xclick	/m/0bt9lr	1	0.000000	0.999219	0.000000	0.998824	1	1	...	0	0	0.508594	0.999219	0.000000	0.478906	0.000000	0.375294	0.720000	0.998824
4	00064d23bf997652	xclick	/m/0bt9lr	1	0.240938	0.906183	0.000000	0.694286	0	0	...	0	0	0.678038	0.906183	0.240938	0.522388	0.000000	0.370000	0.424286	0.694286

5 rows × 21 columns

Preprocess

Rescale the images

!mkdir -p images/resized

%%time
# resize image and save, convert to numpy

img_arr = np.zeros((df.shape[0],128*128*3)) # initialize np.array

for i, f in enumerate(tqdm(df.ImageID)):
    img = Image.open(path+f+'.jpg')
    img_resized = img.resize((128,128))
    img_resized.save("images/resized/"+f+'.jpg', "JPEG", optimize=True)
    img_arr[i] = np.asarray(img_resized, dtype=np.uint8).flatten()

 25%|██▌       | 3263/12966 [04:09<53:44,  3.01it/s]   

Plot the resized and filtered images

# plot random 6 images
fig, ax = plt.subplots(nrows=2, ncols=3, sharex=False, sharey=False,figsize=(15,10))
ax = ax.flatten()

for i,j in enumerate(np.random.choice(df.shape[0], size=6, replace=False)):
    img = mpimg.imread(path+'/resized/'+df.ImageID.values[j]+'.jpg')
    h, w = img.shape[:2]
    coords = df.iloc[j,4:8]
    ax[i].imshow(img)
    ax[i].set_title(df.iloc[j,2])
    ax[i].add_patch(plt.Rectangle((coords[0]*w, coords[2]*h), 
                                  coords[1]*w-coords[0]*w, coords[3]*h-coords[2]*h, 
                                  edgecolor='red', facecolor='none'))

plt.tight_layout()
plt.show()

# encode labels
df['Label'] = (df.LabelName == 'dog').astype(np.uint8)

Checkpoint and Save data

mkdir -p data

np.save('data/img.npy', img_arr.astype(np.uint8))
np.save('data/y_label.npy', df.Label.values)
np.save('data/y_bbox.npy', df[['XMin', 'YMin', 'XMax', 'YMax']].values.astype(np.float32))

Baseline in SKLearn

Metrics for evaluation:

• Accuracy - the ratio of correction predictions to the total predictions

  Accuracy = (True Cats + True Dogs) / (True Cats + False Cats + True Dogs + False Dogs)

  Confusion matrix – Gives us F1 score = 2precision recall / (precision + recall)

  Recall is the ratio of true positives to all actual positives

  Recall = True positives / (True positives + False negatives)

  Precision is the ratio of true positives to all predicted positives

  Precision = True positives / (True positives + False positives)

• F1 score metric weights both recall and precision equally and thereby a higher value of recall and precision will ensure superior performance of the model

Loss function

• Cross entropy loss function for classification and MSE for regression were used as loss function for the neural network

Load data

X = np.load('data/img.npy', allow_pickle=True)
y_label = np.load('data/y_label.npy', allow_pickle=True)
y_bbox = np.load('data/y_bbox.npy', allow_pickle=True)

idx_to_label = {1:'dog', 0:'cat'} # encoder

Double check that it loaded correctly

# plot random 6 images
fig, ax = plt.subplots(nrows=2, ncols=3, sharex=False, sharey=False,figsize=(15,10))
ax = ax.flatten()

for i,j in enumerate(np.random.choice(X.shape[0], size=6, replace=False)):
    coords = y_bbox[j] * 128
    ax[i].imshow(X[j].reshape(128,128,3))
    ax[i].set_title(idx_to_label[y_label[j]])
    ax[i].add_patch(plt.Rectangle((coords[0], coords[1]), 
                                  coords[2]-coords[0], coords[3]-coords[1], 
                                  edgecolor='red', facecolor='none'))

plt.tight_layout()
plt.show()

Homegrown CXE + MSE

• We have built a homegrown logistic regression for image classification which gives the CXE loss at each epoch. Similarly, we have built a homegrown linear regression for boundary box detection which gives the MSE for each epoch.

• The errors from both models are them combines to give us a complex loss function CXE + MSE which was then used to make predictions.

Homegrown Linear Regression

Linear Regression Loss Function

The mean square function formula is as follows : $ \text{MSE}({\mathbf{\theta}}; \mathbf{X}) = \dfrac{1}{m} \sum\limits_{i=1}^{m}{( \hat{y_i} - y_i)^2} $

where $m$ is the number of data points , $\hat{y_i}$ is the predicted value

Split data

import random
random.seed(10)
idxs = random.sample(range(0, X.shape[0]), 500)
X_final = X[idxs]
y_label_final = y_label[idxs]

X_final = X[idxs]
y_bbox_final = y_bbox[idxs]

X_train, X_test, y_train, y_test = train_test_split(X_final, y_bbox_final, test_size=0.01, random_state=27)
X_train, X_valid, y_train, y_valid = train_test_split(X_train, y_train, test_size=0.1, random_state=27)

# np.random.seed(42)
# if np.max(X_train) > 4.:
#     X_train = X_train.astype(np.float32) / 255.
# if np.max(X_test) > 4.:
#     X_test = X_test.astype(np.float32) / 255.
#y_train=y_train.astype(int)
#y_test=y_test.astype(int)

from sklearn.preprocessing import MinMaxScaler, Normalizer
scaler = Normalizer(norm = 'l1')
X_train = scaler.fit_transform(X_train)
X_valid = scaler.transform(X_valid) #Transform test set with the same constants
#X_test = scaler.transform(X_test) #Transform test set with the same constants

Train

#Source - HW -5 
class BasicLinearRegressionHomegrown(object):
    
    def __init__(self, l1_reg = 0.0, l2_reg = 0.01):
        self.coef_ = None       # weight vector
        self.intercept_ = None  # bias term
        self._theta = None      # augmented weight vector, i.e., bias + weights
                                # this allows to treat all decision variables homogeneously
        self.l1_reg = l1_reg
        self.l2_reg = l2_reg
        self.history = {"cost": [], 
                        'val_cost':[],
                        "coef": [], 
                        "intercept": [], 
                        "grad": []}
        
    def _grad(self, X, y):
        """
        Calculate the gradient of the objective function

        Args:
            X(ndarray):        train objects
            y(ndarray):        answers for train objects
        Return:
            gradient(ndarray): analytical gradient vector
        """
        pred = np.dot(X, self._theta)
        error = pred - y
        gradient = 2 * np.dot(error, X) / X.shape[0]
        gradient[1:] += 2 * self.l2_reg * self._theta[1:] + self.l1_reg * np.sign(self._theta[1:])
        return gradient
    
    # full gradient descent, i.e., not stochastic gd
    def _gd(self, X, y, max_iter, X_val, y_val, alpha = 0.003):
        """
        Runs GD and logs error, weigths, gradient at every step

        Args:
            X(ndarray):      train objects
            y(ndarray):      answers for train objects
            max_iter(int):   number of weight updates
            alpha(floar):    step size in direction of gradient
        Return:
            None
        """
        for i in range(max_iter):
            
            self.history["coef"].append(self._theta[1:].copy())
            self.history["intercept"].append(self._theta[0].copy())
            
            mse = self.score(X, y)
            self.history["cost"].append(mse)
            
            if X_val is not None:
                mse = self.score(X_val, y_val)
                self.history["val_cost"].append(mse)
                
            # calculate gradient
            grad = self._grad(X, y)
            self.history["grad"].append(grad)
            
            # do gradient step
            self._theta -= alpha * grad
    
    def fit(self, X, y, max_iter=100, val_data = None):
        """
        Public API for fitting a linear regression model

        Args:
            X(ndarray):      train objects
            y(ndarray):      answers for train objects
            max_iter(int):   number of weight updates
        Return:
            self
        """
        # Augment the data with the bias term.
        # So we can treat the the input variables and the bias term homogeneously 
        # from a vectorization perspective
        X = np.c_[np.ones(X.shape[0]), X]
        # initialize if the first step
        if val_data is not None:
            X_val, y_val = val_data
            X_val = np.c_[np.ones(X_val.shape[0]), X_val]
        else:
            X_val = None
            y_val = None
            
        if self._theta is None:
            self._theta = np.random.rand(X.shape[1])
        
        # do full gradient descent
        self._gd(X, y, max_iter, X_val, y_val)
        
        self.intercept_ = self._theta[0]
        self.coef_ = self._theta[1:]
        return self
        
    def MSE(self, X, y, y_pred):
        error = y - y_pred
        mse = (np.sum(error ** 2)) / X.shape[0] 
        return mse
    
    def score(self, X, y):
        pred = self.predict(X)
        error = y - pred
        mse = (np.sum(error ** 2)) / X.shape[0] 
        return mse
        
    def predict(self, X):
        """
        Make a prediction

        Args:
            X(ndarray):      objects
        Return:
            pred(ndarray):   predictions
        """
        # check whether X has appended bias feature or not
        if X.shape[1] == len(self._theta):
            pred = np.dot(X, self._theta)
        else:
            pred = np.dot(X, self.coef_) + self.intercept_
        #print(pred)
        return pred

model_homegrown = BasicLinearRegressionHomegrown(l1_reg = 0, l2_reg = 0.0)

#Training the model for the 4 boundaries
np.random.seed(42)

model_homegrown.fit(X_train, y_train[:,0], max_iter=2000, val_data=(X_valid, y_valid[:,0])) 
y_pred1 = np.array([model_homegrown.predict(X_train)])
cost1_train = model_homegrown.history["cost"]
cost1_val = model_homegrown.history["val_cost"]

model_homegrown = BasicLinearRegressionHomegrown(l1_reg = 0, l2_reg = 0.0)
model_homegrown.fit(X_train, y_train[:,1], max_iter=2000,val_data=(X_valid, y_valid[:,1]))
y_pred2 = np.array([model_homegrown.predict(X_train)])
cost2_train = model_homegrown.history["cost"]
cost2_val = model_homegrown.history["val_cost"]

model_homegrown = BasicLinearRegressionHomegrown(l1_reg = 0, l2_reg = 0.0)
model_homegrown.fit(X_train, y_train[:,2], max_iter=2000,val_data=(X_valid, y_valid[:,2]))
y_pred3 = np.array([model_homegrown.predict(X_train)])
cost3_train = model_homegrown.history["cost"]
cost3_val = model_homegrown.history["val_cost"]

model_homegrown = BasicLinearRegressionHomegrown(l1_reg = 0, l2_reg = 0.0)
model_homegrown.fit(X_train, y_train[:,3], max_iter=2000,val_data=(X_valid, y_valid[:,3]))
y_pred4 = np.array([model_homegrown.predict(X_train)])
cost4_train = model_homegrown.history["cost"]
cost4_val = model_homegrown.history["val_cost"]

y_pred_test1 = np.array([model_homegrown.predict(X_train)])
y_pred_test2 = np.array([model_homegrown.predict(X_train)])
y_pred_test3 = np.array([model_homegrown.predict(X_train)])
y_pred_test4 = np.array([model_homegrown.predict(X_train)])

y_pred_train = np.concatenate((y_pred1, y_pred2,y_pred3,y_pred4),axis = 0)
y_pred_test = np.concatenate((y_pred_test1, y_pred_test2,y_pred_test3,y_pred_test4),axis = 0)
#y_pred_train
#y_pred_test

print(y_train.shape)
print(y_pred_train.T.shape)
print(y_pred_test.T.shape)

(445, 4)
(445, 4)
(445, 4)

Evaluation

Mean Square Error:

• MSE gives how close our predicted regression line is to the data points
  • MSE = 1/n * Sum(True Value - Predicted value)^^2
  • Where n is the sample size

train_cost = np.sum((np.array(cost1_train), np.array(cost2_train), np.array(cost3_train), np.array(cost4_train)), axis = 0)
val_cost = np.sum((np.array(cost1_val), np.array(cost2_val), np.array(cost3_val), np.array(cost4_val)), axis = 0)

len(model_homegrown.history["val_cost"])

2000

plt.figure(figsize=(20, 8))
plt.suptitle("Homegrown Linear Regression")
plt.subplot(121)
plt.plot(train_cost, label="Train")
plt.plot(val_cost, label="Valid")
plt.legend(loc="upper left")
plt.xlabel("Iteration")
plt.ylabel("Loss")
plt.xlim([0,100])
#plt.ylim([0.0, 0.3])

#plt.subplot(122)
plt.show()

Homegrown implementation of Logistic Regression

Implement a Homegrown Logistic Regression model. Extend the loss function from CXE to CXE + MSE, i.e., make it a complex multitask loss function the resulting model predicts the class and bounding box coordinates at the same time.

from sklearn.preprocessing import StandardScaler

X_train, X_test, y_train, y_test_label = train_test_split(X_final, y_label_final, test_size=0.01, random_state=27)
#s = StandardScaler()
#s.fit_transform(X_train)
#s.transform(X_test)

# Source - HW 7
class LogisticRegressionHomegrown(object):
    
    def __init__(self):
        """
        Constructor for the homgrown Logistic Regression
        
        Args:
            None
        
        Return:
            None
        """
        self.coef_ = None       
        self.intercept_ = None  
        self._theta = None      
                                
        self.history = {"cost": [], 
                        "acc": [], 
                        "val_cost":[], 
                        "val_acc": [],
                        'val_prob':[],
                       "prob":[]}
        
    def _grad(self, X, y):
        """
        Calculates the gradient of the Logistic Regression 
        objective function

        Args:
            X(ndarray):    train objects
            y(ndarray):    answers for train objects
            
        Return:
            grad(ndarray): gradient
        """
        
        n = X.shape[0]

        scores = self._predict_raw(X)

        exp_scores = np.exp(scores)
        probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
 
        probs[range(n),y] -= 1
 
        gradient = np.dot(X.T, probs) / n
        
        return gradient
    
    def _gd(self, X, y, max_iter, alpha, X_val, y_val):

        for i in range(max_iter):
            
            metrics = self.score(X, y)
            self.history["cost"].append(metrics["cost"])
            self.history["acc"].append(metrics["acc"])
            self.history["prob"].append(metrics["prob"])
            
            if X_val is not None:
                metrics_val = self.score(X_val, y_val)
                self.history["val_cost"].append(metrics_val["cost"])
                self.history["val_acc"].append(metrics_val["acc"])
                self.history["val_prob"].append(metrics_val["prob"])

            # gradient
            grad = self._grad(X, y)
            

            self._theta -= alpha * grad
    
    def fit(self, X, y, max_iter=100, alpha=0.05, val_data=None):

        X = np.c_[np.ones(X.shape[0]), X]
        if val_data is not None:
            X_val, y_val = val_data
            X_val = np.c_[np.ones(X_val.shape[0]), X_val]
        else:
            X_val = None
            y_val = None
        
        if self._theta is None:
            self._theta = np.random.rand(X.shape[1], len(np.unique(y)))
        
        
        self._gd(X, y, max_iter, alpha, X_val, y_val)
        
        
        self.intercept_ = self._theta[0]
        self.coef_ = self._theta[1:]
        
    def score(self, X, y):

        n = X.shape[0]

        scores = self._predict_raw(X)
        
        
        exp_scores = np.exp(scores)
        probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
        

        corect_logprobs = -np.log(probs[range(n),y])

        data_loss = np.sum(corect_logprobs) / n
        
        # predictions
        pred = np.argmax(scores, axis=1)
        # accuracy
        acc = accuracy_score(y, pred)

        metrics = {"acc": acc, "cost": data_loss, 'prob':corect_logprobs}
        
        return metrics
        
    def _predict_raw(self, X):

        if X.shape[1] == len(self._theta):
            scores = np.dot(X, self._theta)
        else:
            scores = np.dot(X, self.coef_) + self.intercept_
        return scores
    
    def predict(self, X):
        """
        Predicts class for each object in X
        
        Args:
            X(ndarray):      objects
        
        Return:
            pred(ndarray):   class for each object
        """
        # get scores for each class
        scores = self._predict_raw(X)
       
        pred = np.argmax(scores, axis=1)
        return pred

X_train, X_test, y_train, y_test_label = train_test_split(X_final, y_label_final, test_size=0.01, random_state=27)
np.random.seed(42)
if np.max(X_train) > 4.:
    X_train = X_train.astype(np.float32) / 255.
if np.max(X_test) > 4.:
    X_test = X_test.astype(np.float32) / 255.
y_train=y_train.astype(int)
y_test=y_test.astype(int)

class FixedLogisticRegressionHomegrown(LogisticRegressionHomegrown):
    
    def __init__(self):
        # call the constructor of the parent class
        super(FixedLogisticRegressionHomegrown, self).__init__()
        
    
    def _predict_raw(self, X):

        # check whether X has appended bias feature or not
        if X.shape[1] == len(self._theta):
            scores = np.dot(X, self._theta)
        else:
            scores = np.dot(X, self.coef_) + self.intercept_
        
        # normalize raw scores to prevent overflow
        scores -= np.max(scores, axis=1, keepdims=True)
        
        return scores

 

model_lr_homegrown_fixed = FixedLogisticRegressionHomegrown()

#Training model
model_lr_homegrown_fixed.fit(X_train, y_train, max_iter=2000, alpha=0.05, val_data=(X_test, y_test_label))

len(model_lr_homegrown_fixed.history["cost"])

2000

plt.figure(figsize=(20, 8))
plt.suptitle("Homegrown Logistic Regression")
#plt.subplot(121)
plt.plot(model_lr_homegrown_fixed.history["cost"], label="Train")
plt.plot(model_lr_homegrown_fixed.history["val_cost"], label="Test")
plt.legend(loc="upper left")
plt.xlabel("Iteration")
plt.ylabel("Loss")
plt.show()

y_pred_test = model_lr_homegrown_fixed.predict(X_test)

Combined MSE and CXE loss.

• MSE calculated for boundary detection is combined here with CXE value. For each image data here we are calculating seperate MSE and CXE values and combining those will provide loss.

from sklearn.metrics import log_loss
def comb_MSE_CXE(y1,y2):
    sum_error = []
    for i in range(0, len(y1)):
        #y1_pred = model.predict([x[i]])
        #print(y1_pred)
        #y2_pred = y2[i]
        mse = y1[i]
        cxe = y2[i]
        sum1 = cxe + mse
        sum_error.append(sum1)
        #print(mse, cxe)
    return sum_error

sum_error_train = comb_MSE_CXE(train_cost, np.mean(model_lr_homegrown_fixed.history['prob'], axis = 1))

sum_error_valid = comb_MSE_CXE(val_cost, np.mean(model_lr_homegrown_fixed.history['val_prob'], axis = 1))

• Plotting data for combined values of MSE and CXE.

#Plotting combined loss
plt.figure(figsize=(20, 8))
plt.suptitle("Combined MSE and CXE")
#plt.subplot(121)
plt.plot(sum_error_train, label="Train")
plt.plot(sum_error_valid, label="Valid")
plt.legend(loc="upper left")
plt.xlabel("Iteration")
plt.ylabel("Loss")

plt.show()

Result:

• Combined loss decreased steadily at each epoch
• After around 100 epochs, the validation loss started to decrease at a higher rate

Multi Layered Perceptron

We will be using both sequential and class definition (OOP API) to build neural networks.

• Sequential neural networks:

  • In this method, we will pass all our connecting layers in the sequential class constructor itself

• Class Definition (OOP API):

  • To accomplish this in PyTorch you define a Model as a subclasses of torch.nn.Module. This model will include the following methods:

    -init method : Initialize the layers we want to use. We have to specify the sizes of our network in this method. We will also add all the layers we want to use in our network in this method

    -forward method : We will specify the connections between the layers in this method. We will use the layers you already initialized in the init method, in order to re-use the same layer for each forward pass of data

• Classification:

  • In both the above cases, we have split the data into train, validation, and test and used 30 epochs to help in the convergence process

    • For image detection, we have used 1 input layer, 2 hidden layers and one output layer.
    • ReLU has been used as the activation function and cross-entropy is the loss function.
    • In addition to this, we have experimented with dropout layers.
    • We are using Adam optimizer as an optimizer for convergence

    -Regression:

    • We have split the data into train, validation, and tests and used 50 epochs to help in the convergence process
    • We have used 1 input layer, 2 hidden layer and 1 output layer to make the prediction for our four boundaries -We have use tanh as our activation function -Mean square is our loss function and we have experimented using stochastic gradient descent and adam as an optimizer for convergence

Image classification

Sequential neural network classifier

• Loss function used – Cross Entropy Loss function

• We used a one input layer, 2 hidden layers, and an output layer in our neural network

• ReLU was used as the activation function

• We also used the following to experiments to avoid overfitting improve accuracy:

  • Drop out layer With p values of 0.1 and 0.2

  • Batch normalization after output of hidden layer

  • We have used Adam optimizer with a learning rate of .0002

  • We also used a weight decay value of 1e_4 which will provide us with L2 regularization

A similar process was followed for OOP as well

import pandas as pd
import numpy as np
from sklearn.metrics import classification_report, confusion_matrix
from torch.optim import Adam
import matplotlib.pyplot as plt
import seaborn as sns

# Setting seeds to try and ensure we have the same results - this is not guaranteed across PyTorch releases.
import torch
torch.manual_seed(0)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False

import torch.nn as nn
import torch.nn.functional as F
from torchvision import datasets, transforms, models
from torch.utils.data import DataLoader, SubsetRandomSampler
import random
from os import listdir
from shutil import copyfile

import tensorflow as tf 
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from os import makedirs

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
device

device(type='cpu')

from google.colab import drive drive.mount('/content/drive')

!ls "/content/drive/My Drive/Colab Notebooks"

random.seed(42)
X = np.load('data/img.npy', allow_pickle=True)
y_label = np.load('data/y_label.npy', allow_pickle=True)
y_bbox = np.load('data/y_bbox.npy', allow_pickle=True)

print(X.shape, y_label.shape)

(12966, 49152) (12966,)

#creating directories for test and train data set
dataset_home = 'cat_vs_dog/'
subdirs = ['train/', 'test/']
for subdir in subdirs:
	# create label subdirectories
	labeldirs = ['dogs/', 'cats/']
	for labldir in labeldirs:
		newdir = dataset_home + subdir + labldir
		makedirs(newdir, exist_ok=True)

#load the csv file
df = pd.read_csv("cadod.csv")
df.head()

	ImageID	Source	LabelName	Confidence	XMin	XMax	YMin	YMax	IsOccluded	IsTruncated	...	IsDepiction	IsInside	XClick1X	XClick2X	XClick3X	XClick4X	XClick1Y	XClick2Y	XClick3Y	XClick4Y
0	0000b9fcba019d36	xclick	/m/0bt9lr	1	0.165000	0.903750	0.268333	0.998333	1	1	...	0	0	0.636250	0.903750	0.748750	0.165000	0.268333	0.506667	0.998333	0.661667
1	0000cb13febe0138	xclick	/m/0bt9lr	1	0.000000	0.651875	0.000000	0.999062	1	1	...	0	0	0.312500	0.000000	0.317500	0.651875	0.000000	0.410882	0.999062	0.999062
2	0005a9520eb22c19	xclick	/m/0bt9lr	1	0.094167	0.611667	0.055626	0.998736	1	1	...	0	0	0.487500	0.611667	0.243333	0.094167	0.055626	0.226296	0.998736	0.305942
3	0006303f02219b07	xclick	/m/0bt9lr	1	0.000000	0.999219	0.000000	0.998824	1	1	...	0	0	0.508594	0.999219	0.000000	0.478906	0.000000	0.375294	0.720000	0.998824
4	00064d23bf997652	xclick	/m/0bt9lr	1	0.240938	0.906183	0.000000	0.694286	0	0	...	0	0	0.678038	0.906183	0.240938	0.522388	0.000000	0.370000	0.424286	0.694286

5 rows × 21 columns

df.LabelName.replace({'/m/01yrx':'cat', '/m/0bt9lr':'dog'}, inplace=True)

df.head()

	ImageID	Source	LabelName	Confidence	XMin	XMax	YMin	YMax	IsOccluded	IsTruncated	...	IsDepiction	IsInside	XClick1X	XClick2X	XClick3X	XClick4X	XClick1Y	XClick2Y	XClick3Y	XClick4Y
0	0000b9fcba019d36	xclick	dog	1	0.165000	0.903750	0.268333	0.998333	1	1	...	0	0	0.636250	0.903750	0.748750	0.165000	0.268333	0.506667	0.998333	0.661667
1	0000cb13febe0138	xclick	dog	1	0.000000	0.651875	0.000000	0.999062	1	1	...	0	0	0.312500	0.000000	0.317500	0.651875	0.000000	0.410882	0.999062	0.999062
2	0005a9520eb22c19	xclick	dog	1	0.094167	0.611667	0.055626	0.998736	1	1	...	0	0	0.487500	0.611667	0.243333	0.094167	0.055626	0.226296	0.998736	0.305942
3	0006303f02219b07	xclick	dog	1	0.000000	0.999219	0.000000	0.998824	1	1	...	0	0	0.508594	0.999219	0.000000	0.478906	0.000000	0.375294	0.720000	0.998824
4	00064d23bf997652	xclick	dog	1	0.240938	0.906183	0.000000	0.694286	0	0	...	0	0	0.678038	0.906183	0.240938	0.522388	0.000000	0.370000	0.424286	0.694286

5 rows × 21 columns

dog_list = df[df.LabelName == 'dog']['ImageID']
cat_list = df[df.LabelName == 'cat']['ImageID']
#list(dog_list)

# moving images to test and train folder

random.seed(10)
# define ratio of pictures to use for test
test_ratio = 0.20
count_c = 0
count_d = 0

# copy training dataset images into subdirectories
src_directory = 'cadod/'
for file in listdir(src_directory):
    #print(file.replace('.jpg','').replace('._','') in list(cat_list))
    #print(file.replace('.jpg','').replace('._','') in list(dog_list))
    src = src_directory + '/' + file
    dst_dir = 'train/'
    if random.random() < test_ratio:
        dst_dir = 'test/'
    if file.replace('.jpg','').replace('._','') in list(cat_list) and count_c < 2000:
        dst = dataset_home + dst_dir + 'cats/'  + file
        count_c +=1
        copyfile(src, dst)
    elif file.replace('.jpg','').replace('._','') in list(dog_list) and count_d < 2000:
        dst = dataset_home + dst_dir + 'dogs/'  + file
        count_d +=1
        copyfile(src, dst)

from tensorflow.keras import models
from tensorflow.keras.applications import *
from tensorflow.keras import layers

Training MultiLayer Perceptron

Image classification

Sequential Neural Network Model

expLog1 = pd.DataFrame(columns=["exp_name", 
                               "Train Acc", 
                               "Valid Acc",
                               "Test  Acc",
                              ])

#Using nn.sequential to build a muti layer perceptron
from torch.optim import Adam
model = nn.Sequential(
nn.Flatten(),
nn.Linear(128*128*3, 128),
nn.ReLU(),
nn.BatchNorm1d(128),
 nn.Linear(128, 64),
nn.ReLU(),
nn.BatchNorm1d(64),   
    nn.Dropout(0.2),
    nn.Linear(64,2),
    nn.Softmax(dim = 1)
)

criterion = nn.CrossEntropyLoss() #loss function
optimizer = torch.optim.Adam(model.parameters(), lr=0.0003, weight_decay = 1e-3) # Optimizer with reularization
scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones = [500,1000,1500], gamma = 0.5)

accuracy_stats = {
    'train': [],
    "val": []
}
loss_stats = {
    'train': [],
    "val": []
}

#Calcualtion if accuracy
def binary_acc(y_pred, y_test):
    y_pred_tag = torch.log_softmax(y_pred, dim = 1)
    _, y_pred_tags = torch.max(y_pred_tag, dim = 1)    
    correct_results_sum = (y_pred_tags == y_test).sum().float()    
    acc = correct_results_sum/y_test.shape[0]
    acc = torch.round(acc * 100)    
    return acc

model = model.to(device)
#optimizer = Adam(filter(lambda p: p.requires_grad, model.parameters()))

num_epochs = 25

#Test set 
for e in range(num_epochs):
    cum_epoch_loss = 0
    cum_acc = 0
    batch_loss = 0
    model.train()
    for batch, (images, labels) in enumerate(trainloader,1):

        images = images.to(device)
        labels = labels.to(device)
        optimizer.zero_grad()
        label_pred = model(images).squeeze() #Fitting the model
        loss = criterion(label_pred, labels) #Calculating loss
        acc = binary_acc(label_pred, labels) #Calculating accuracy
        loss.backward() #back propagting the loss
        optimizer.step() #Modify weights and bias based on the loss
        batch_loss += loss.item()
        cum_acc += acc.item()
        scheduler.step()
       
    #Evaluating mode on validation set
    with torch.no_grad():
        model.eval()
        val_epoch_loss = 0
        val_epoch_acc = 0
        for batch, (X_val_batch, y_val_batch) in enumerate(valloader,1):
            X_val_batch, y_val_batch = X_val_batch.to(device), y_val_batch.to(device)            
            y_val_pred = model(X_val_batch).squeeze()
            #y_val_pred = torch.unsqueeze(y_val_pred, 0)            
            val_loss = criterion(y_val_pred, y_val_batch)
            val_acc = binary_acc(y_val_pred, y_val_batch)            
            val_epoch_loss += val_loss.item()
            val_epoch_acc += val_acc.item()
        #print(f'Epoch({e}/{num_epochs} : Train loss : {loss.item()} : Train Accuracy : {acc.item()} : Valid loss : {val_loss.item()} : Valid Accuracy : {val_acc.item()}')
    
    loss_stats['train'].append(batch_loss/len(trainloader))
    loss_stats['val'].append(val_epoch_loss/len(valloader))
    accuracy_stats['train'].append(cum_acc/len(trainloader))
    accuracy_stats['val'].append(val_epoch_acc/len(valloader))

    
    print(f'Epoch({e}/{num_epochs})')
    print(f'Training loss : {batch_loss/len(trainloader)}')  
    print(f'Training accuracy : {cum_acc/len(trainloader)}')  
    print(f'Validation loss : {val_epoch_loss/len(valloader)}')  
    print(f'Validation accuracy : {val_epoch_acc/len(valloader)}') 
    
        

    

Epoch(0/25)
Training loss : 0.7125498804640262
Training accuracy : 50.1063829787234
Validation loss : 0.6871637397342258
Validation accuracy : 54.62222222222222
Epoch(1/25)
Training loss : 0.693171889223951
Training accuracy : 53.0
Validation loss : 0.684539356496599
Validation accuracy : 54.6
Epoch(2/25)
Training loss : 0.6835525986996103
Training accuracy : 55.829787234042556
Validation loss : 0.6752279546525743
Validation accuracy : 58.355555555555554
Epoch(3/25)
Training loss : 0.68639815107305
Training accuracy : 55.61702127659574
Validation loss : 0.6795152836375766
Validation accuracy : 55.44444444444444
Epoch(4/25)
Training loss : 0.6779653873849423
Training accuracy : 57.06382978723404
Validation loss : 0.6744380593299866
Validation accuracy : 56.77777777777778
Epoch(5/25)
Training loss : 0.6795218282557548
Training accuracy : 56.723404255319146
Validation loss : 0.6762206421958076
Validation accuracy : 56.86666666666667
Epoch(6/25)
Training loss : 0.6784659750918125
Training accuracy : 56.787234042553195
Validation loss : 0.6791158636411031
Validation accuracy : 56.62222222222222
Epoch(7/25)
Training loss : 0.672343799408446
Training accuracy : 57.93617021276596
Validation loss : 0.6755600823296442
Validation accuracy : 57.0
Epoch(8/25)
Training loss : 0.6696452358935742
Training accuracy : 59.234042553191486
Validation loss : 0.688512482908037
Validation accuracy : 54.4
Epoch(9/25)
Training loss : 0.6769240726815894
Training accuracy : 57.42553191489362
Validation loss : 0.6722669959068298
Validation accuracy : 58.4
Epoch(10/25)
Training loss : 0.6671230590089838
Training accuracy : 59.04255319148936
Validation loss : 0.6699104693200854
Validation accuracy : 59.15555555555556
Epoch(11/25)
Training loss : 0.670653429437191
Training accuracy : 59.191489361702125
Validation loss : 0.6782884531550937
Validation accuracy : 57.577777777777776
Epoch(12/25)
Training loss : 0.6701804792627375
Training accuracy : 58.93617021276596
Validation loss : 0.6720785432391696
Validation accuracy : 58.955555555555556
Epoch(13/25)
Training loss : 0.660548395298897
Training accuracy : 60.638297872340424
Validation loss : 0.6715567986170451
Validation accuracy : 56.111111111111114
Epoch(14/25)
Training loss : 0.661698119437441
Training accuracy : 60.4468085106383
Validation loss : 0.6743685682614644
Validation accuracy : 57.888888888888886
Epoch(15/25)
Training loss : 0.6616999354768307
Training accuracy : 60.829787234042556
Validation loss : 0.6778645912806193
Validation accuracy : 57.06666666666667
Epoch(16/25)
Training loss : 0.6606974335426979
Training accuracy : 61.765957446808514
Validation loss : 0.6738517840703329
Validation accuracy : 58.6
Epoch(17/25)
Training loss : 0.660804180388755
Training accuracy : 60.38297872340426
Validation loss : 0.6713690214686924
Validation accuracy : 59.48888888888889
Epoch(18/25)
Training loss : 0.6543172762749043
Training accuracy : 61.40425531914894
Validation loss : 0.6753527283668518
Validation accuracy : 59.111111111111114
Epoch(19/25)
Training loss : 0.6574419052042859
Training accuracy : 61.48936170212766
Validation loss : 0.6824199557304382
Validation accuracy : 56.266666666666666
Epoch(20/25)
Training loss : 0.6570355575135414
Training accuracy : 61.191489361702125
Validation loss : 0.6700100117259555
Validation accuracy : 58.86666666666667
Epoch(21/25)
Training loss : 0.6556217530940441
Training accuracy : 61.829787234042556
Validation loss : 0.669213142659929
Validation accuracy : 58.644444444444446
Epoch(22/25)
Training loss : 0.6551772736488505
Training accuracy : 61.319148936170215
Validation loss : 0.6726504670249092
Validation accuracy : 58.2
Epoch(23/25)
Training loss : 0.6510694179129093
Training accuracy : 63.212765957446805
Validation loss : 0.6740733835432264
Validation accuracy : 57.422222222222224
Epoch(24/25)
Training loss : 0.6496258428756226
Training accuracy : 62.87234042553192
Validation loss : 0.6766399105389913
Validation accuracy : 56.733333333333334

#Plotting testing and validation accuracies
train_val_acc_df = pd.DataFrame.from_dict(accuracy_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})
train_val_loss_df = pd.DataFrame.from_dict(loss_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})

fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(30,10))

sns.lineplot(data=train_val_acc_df, x = "epochs", y="value", hue="variable", ax = axes[0]).set_title('Accuracy/Epoch')
sns.lineplot(data=train_val_loss_df, x = "epochs", y="value", hue="variable", ax = axes[1]).set_title('Loss/Epoch')
plt.show()

#Testing the model
y_pred_list = []
y_true_list = []
model.eval()
with torch.no_grad():
    num_correct = 0
    total = 0
    #set_trace()
    for batch, (images, labels) in enumerate(testloader,1):
        pred = model(images)
        #output = torch.exp(logps)
        pred = torch.argmax(pred, 1)
        y_pred_list.append(pred.numpy())
        y_true_list.append(labels.numpy())
        total += labels.size(0)
        num_correct += (pred == labels).sum().item()
        
        print(f'Batch ({batch}/{len(testloader)})')


    print(f'Accuracy of the model on {total} test images: {num_correct * 100 / total}% ')

Batch (1/49)
Batch (2/49)
Batch (3/49)
Batch (4/49)
Batch (5/49)
Batch (6/49)
Batch (7/49)
Batch (8/49)
Batch (9/49)
Batch (10/49)
Batch (11/49)
Batch (12/49)
Batch (13/49)
Batch (14/49)
Batch (15/49)
Batch (16/49)
Batch (17/49)
Batch (18/49)
Batch (19/49)
Batch (20/49)
Batch (21/49)
Batch (22/49)
Batch (23/49)
Batch (24/49)
Batch (25/49)
Batch (26/49)
Batch (27/49)
Batch (28/49)
Batch (29/49)
Batch (30/49)
Batch (31/49)
Batch (32/49)
Batch (33/49)
Batch (34/49)
Batch (35/49)
Batch (36/49)
Batch (37/49)
Batch (38/49)
Batch (39/49)
Batch (40/49)
Batch (41/49)
Batch (42/49)
Batch (43/49)
Batch (44/49)
Batch (45/49)
Batch (46/49)
Batch (47/49)
Batch (48/49)
Batch (49/49)
Accuracy of the model on 781 test images: 57.87451984635083% 

#Calulating the number of right and wring prediction to get the
#classification report and confusion matrix
y_pred_list = []
y_true_list = []
with torch.no_grad():
    for batch, (images, labels) in enumerate(testloader,1):      
        y_test_pred = model(images)
        _, y_pred_tag = torch.max(y_test_pred, dim = 1)        
        y_pred_list = [*y_pred_list,*y_pred_tag.cpu().numpy()]
        y_true_list = [*y_true_list,*labels.cpu().numpy()]

#Logging the experiment
exp_name = f"Sequential Neural Network"
expLog1.loc[0,:4] = [f"{exp_name}"] + list(np.round(
               [accuracy_stats['train'][-1], 
                accuracy_stats['val'][-1],
                (num_correct * 100 / total)],3))
expLog1

/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py:719: FutureWarning: Slicing a positional slice with .loc is not supported, and will raise TypeError in a future version.  Use .loc with labels or .iloc with positions instead.
  indexer = self._get_setitem_indexer(key)

	exp_name	Train Acc	Valid Acc	Test Acc
0	Sequential Neural Network	62.872	56.733	57.875

y_pred_list_1 = np.array(y_pred_list).T
y_true_list_1 = np.array(y_true_list).T
#y_pred_list_1

print(classification_report(y_true_list_1, y_pred_list_1))

              precision    recall  f1-score   support

           0       0.61      0.50      0.55       402
           1       0.55      0.67      0.61       379

    accuracy                           0.58       781
   macro avg       0.58      0.58      0.58       781
weighted avg       0.58      0.58      0.58       781

print(confusion_matrix(y_true_list_1, y_pred_list_1))

[[199 203]
 [126 253]]

#Plot confusiion matrix
idx2class = {v: k for k, v in train_it.class_to_idx.items()}
confusion_matrix_df = pd.DataFrame(confusion_matrix(y_true_list_1, y_pred_list_1)).rename(columns=idx2class, index=idx2class)
fig, ax = plt.subplots(figsize=(7,5))         
sns.heatmap(confusion_matrix_df, annot=True, ax=ax, cmap = 'Blues', fmt = 'g')
plt.show()

PyTorch OOP API Neural network

expLog = pd.DataFrame(columns=["exp_name", 
                               "Train Acc", 
                               "Valid Acc",
                               "Test  Acc",
                              ])

Experiments

Without Data Augmentation

#Normalizing image and converting to tensor

mean = [0.485, 0.456, 0.406]
std = [0.229, 0.224, 0.225]

transform_test = transforms.Compose([
    #transforms.ToPILImage(),             
    transforms.Resize((128, 128)),
    transforms.ToTensor(),
    transforms.Normalize(mean=mean, std=std)
    ])
transform_train = transforms.Compose([
    #transforms.ToPILImage(),             
    transforms.Resize((128, 128)),
    transforms.ToTensor(),
    transforms.Normalize(mean=mean, std=std)
    #transforms.RandomAutocontrast()
    
    ])

#Loading the image
train_it = datasets.ImageFolder('cat_vs_dog/train/', transform=transform_train)
test_it = datasets.ImageFolder('cat_vs_dog/test/', transform=transform_test)

dataset_size = len(train_it)
dataset_indices = list(range(dataset_size))
np.random.shuffle(dataset_indices)

dataset_size

3219

train_set, val_set = torch.utils.data.random_split(train_it, [2500, 719]) 

#splitting data into train, validation and test
trainloader = DataLoader(train_set, batch_size=64, shuffle=True)
valloader = DataLoader(val_set, shuffle=True, batch_size=16)
testloader = DataLoader(test_it, batch_size=32, shuffle=True)

for images, labels in trainloader:
    print(images.size(), labels.size())
    print(labels)
    break

torch.Size([64, 3, 128, 128]) torch.Size([64])
tensor([1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1,
        0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1,
        0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 1])

#defining neural network layers
class cadod(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(128*128*3, 128) # input layer
     
        self.fc4 = nn.Linear(128,64)
        self.fc5 = nn.Linear(64,2)
        self.relu = nn.ReLU() #activation function
        
        self.norm2 = nn.BatchNorm1d(512) #normalizing the output
        self.norm3 = nn.BatchNorm1d(128)
        self.norm4 = nn.BatchNorm1d(64)


    def forward(self, x):
        x = x.view(x.shape[0], -1)
        x = F.relu(self.fc1(x))

        x = self.norm3(x)
        x = F.relu(self.fc4(x))
        x = self.norm4(x)
        #x =self.dropout(x)
        x = self.fc5(x) 
        x = F.softmax(x, dim = 1) # using softmax as activation layer for output
        return x

model = cadod()
criterion = nn.CrossEntropyLoss() #Loss function
optimizer = torch.optim.Adam(model.parameters(), lr=0.0002) #optimizer with learning rate of 0.0002
#scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones = [500,1000,1500], gamma = 0.5)

#calculation of accuracy
def binary_acc(y_pred, y_test):
    y_pred_tag = torch.log_softmax(y_pred, dim = 1)
    _, y_pred_tags = torch.max(y_pred_tag, dim = 1)    
    correct_results_sum = (y_pred_tags == y_test).sum().float()    
    acc = correct_results_sum/y_test.shape[0]
    acc = torch.round(acc * 100)    
    return acc

accuracy_stats = {
    'train': [],
    "val": []
}
loss_stats = {
    'train': [],
    "val": []
}

model = model.to(device)
#optimizer = Adam(filter(lambda p: p.requires_grad, model.parameters()))

num_epochs = 50


for e in range(num_epochs):
    cum_epoch_loss = 0
    cum_acc = 0
    batch_loss = 0
    model.train()
    for batch, (images, labels) in enumerate(trainloader,1):

        images = images.to(device)
        labels = labels.to(device)
        optimizer.zero_grad()
        label_pred = model(images).squeeze() #fitting the model

        loss = criterion(label_pred, labels) #calculation of loss
        acc = binary_acc(label_pred, labels) #calculation of accuracy
        loss.backward() #back propagation of loss
        optimizer.step() #Optimizing the weights and bias based on the loss value
        batch_loss += loss.item() #calculating cumulative loss
        cum_acc += acc.item() #calculating cumulative accuracy
        
    #using validation set to see if our model took the step in the right direction
    with torch.no_grad():
        model.eval()
        val_epoch_loss = 0
        val_epoch_acc = 0
        for batch, (X_val_batch, y_val_batch) in enumerate(valloader,1):
            X_val_batch, y_val_batch = X_val_batch.to(device), y_val_batch.to(device)            
            y_val_pred = model(X_val_batch).squeeze()
                       
            val_loss = criterion(y_val_pred, y_val_batch)
            val_acc = binary_acc(y_val_pred, y_val_batch)            
            val_epoch_loss += val_loss.item()
            val_epoch_acc += val_acc.item()
        #print(f'Epoch({e}/{num_epochs} : Train loss : {loss.item()} : Train Accuracy : {acc.item()} : Valid loss : {val_loss.item()} : Valid Accuracy : {val_acc.item()}')
    
    loss_stats['train'].append(batch_loss/len(trainloader))
    loss_stats['val'].append(val_epoch_loss/len(valloader))
    accuracy_stats['train'].append(cum_acc/len(trainloader))
    accuracy_stats['val'].append(val_epoch_acc/len(valloader))
    
    print(f'Epoch({e}/{num_epochs})')
    print(f'Training loss : {batch_loss/len(trainloader)}')  
    print(f'Training accuracy : {cum_acc/len(trainloader)}')  
    print(f'Validation loss : {val_epoch_loss/len(valloader)}')  
    print(f'Validation accuracy : {val_epoch_acc/len(valloader)}') 
    

Epoch(0/50)
Training loss : 0.7094970524311066
Training accuracy : 51.15
Validation loss : 0.68065847158432
Validation accuracy : 54.888888888888886
Epoch(1/50)
Training loss : 0.6562335282564163
Training accuracy : 63.15
Validation loss : 0.6780192547374302
Validation accuracy : 56.266666666666666
Epoch(2/50)
Training loss : 0.6370158970355988
Training accuracy : 66.475
Validation loss : 0.674514098962148
Validation accuracy : 59.044444444444444
Epoch(3/50)
Training loss : 0.6026391848921776
Training accuracy : 72.95
Validation loss : 0.6753588464525011
Validation accuracy : 56.44444444444444
Epoch(4/50)
Training loss : 0.5662027284502983
Training accuracy : 77.475
Validation loss : 0.6777131835619609
Validation accuracy : 57.8
Epoch(5/50)
Training loss : 0.5436086222529412
Training accuracy : 81.075
Validation loss : 0.6880022565523783
Validation accuracy : 55.8
Epoch(6/50)
Training loss : 0.5347298622131348
Training accuracy : 81.325
Validation loss : 0.6894654247495863
Validation accuracy : 59.77777777777778
Epoch(7/50)
Training loss : 0.4901034340262413
Training accuracy : 86.425
Validation loss : 0.6969042460123698
Validation accuracy : 55.977777777777774
Epoch(8/50)
Training loss : 0.47560790106654166
Training accuracy : 87.7
Validation loss : 0.7016184409459432
Validation accuracy : 54.2
Epoch(9/50)
Training loss : 0.48551381528377535
Training accuracy : 85.75
Validation loss : 0.7075383941332499
Validation accuracy : 54.15555555555556
Epoch(10/50)
Training loss : 0.45131670460104945
Training accuracy : 90.425
Validation loss : 0.7021801696883307
Validation accuracy : 55.84444444444444
Epoch(11/50)
Training loss : 0.4356138601899147
Training accuracy : 91.55
Validation loss : 0.7082754757669237
Validation accuracy : 55.51111111111111
Epoch(12/50)
Training loss : 0.4608717389404774
Training accuracy : 88.55
Validation loss : 0.7146799551116095
Validation accuracy : 55.68888888888889
Epoch(13/50)
Training loss : 0.42749454379081725
Training accuracy : 92.65
Validation loss : 0.706292392147912
Validation accuracy : 56.31111111111111
Epoch(14/50)
Training loss : 0.4364430636167526
Training accuracy : 90.125
Validation loss : 0.7096594267421299
Validation accuracy : 55.022222222222226
Epoch(15/50)
Training loss : 0.42983848676085473
Training accuracy : 90.725
Validation loss : 0.708766500155131
Validation accuracy : 55.28888888888889
Epoch(16/50)
Training loss : 0.40660866275429725
Training accuracy : 94.375
Validation loss : 0.7068084875742594
Validation accuracy : 55.266666666666666
Epoch(17/50)
Training loss : 0.39544065818190577
Training accuracy : 94.375
Validation loss : 0.7234642995728386
Validation accuracy : 54.022222222222226
Epoch(18/50)
Training loss : 0.4116306290030479
Training accuracy : 92.85
Validation loss : 0.7316117458873325
Validation accuracy : 51.62222222222222
Epoch(19/50)
Training loss : 0.3846778951585293
Training accuracy : 95.7
Validation loss : 0.733617545498742
Validation accuracy : 51.888888888888886
Epoch(20/50)
Training loss : 0.3806115217506886
Training accuracy : 95.45
Validation loss : 0.7261078556378683
Validation accuracy : 51.977777777777774
Epoch(21/50)
Training loss : 0.38025286719202994
Training accuracy : 95.375
Validation loss : 0.7231230530473921
Validation accuracy : 53.24444444444445
Epoch(22/50)
Training loss : 0.3765911430120468
Training accuracy : 95.6
Validation loss : 0.7242089245054457
Validation accuracy : 52.955555555555556
Epoch(23/50)
Training loss : 0.36151179745793344
Training accuracy : 96.675
Validation loss : 0.7206950320137872
Validation accuracy : 54.82222222222222
Epoch(24/50)
Training loss : 0.36770726889371874
Training accuracy : 96.1
Validation loss : 0.7277222262488471
Validation accuracy : 53.93333333333333
Epoch(25/50)
Training loss : 0.3990778349339962
Training accuracy : 93.15
Validation loss : 0.7017573760615454
Validation accuracy : 56.93333333333333
Epoch(26/50)
Training loss : 0.3737898744642735
Training accuracy : 95.95
Validation loss : 0.7303526653183832
Validation accuracy : 52.8
Epoch(27/50)
Training loss : 0.4106456212699413
Training accuracy : 91.2
Validation loss : 0.708319079875946
Validation accuracy : 56.955555555555556
Epoch(28/50)
Training loss : 0.38948198705911635
Training accuracy : 94.75
Validation loss : 0.7306037863095601
Validation accuracy : 53.06666666666667
Epoch(29/50)
Training loss : 0.376045011729002
Training accuracy : 95.9
Validation loss : 0.7133931279182434
Validation accuracy : 55.82222222222222
Epoch(30/50)
Training loss : 0.3787900499999523
Training accuracy : 95.25
Validation loss : 0.7281713869836596
Validation accuracy : 55.84444444444444
Epoch(31/50)
Training loss : 0.381504375487566
Training accuracy : 94.95
Validation loss : 0.7082713550991482
Validation accuracy : 56.62222222222222
Epoch(32/50)
Training loss : 0.36115402579307554
Training accuracy : 96.675
Validation loss : 0.7279562201764849
Validation accuracy : 53.62222222222222
Epoch(33/50)
Training loss : 0.36100330725312235
Training accuracy : 96.7
Validation loss : 0.7206439077854156
Validation accuracy : 54.666666666666664
Epoch(34/50)
Training loss : 0.34364324510097505
Training accuracy : 98.4
Validation loss : 0.734596930609809
Validation accuracy : 53.93333333333333
Epoch(35/50)
Training loss : 0.3596883825957775
Training accuracy : 96.25
Validation loss : 0.7396335138214959
Validation accuracy : 51.13333333333333
Epoch(36/50)
Training loss : 0.3527286686003208
Training accuracy : 96.975
Validation loss : 0.7279335227277544
Validation accuracy : 54.48888888888889
Epoch(37/50)
Training loss : 0.3458149656653404
Training accuracy : 97.6
Validation loss : 0.7156677325566609
Validation accuracy : 56.93333333333333
Epoch(38/50)
Training loss : 0.3760900966823101
Training accuracy : 94.55
Validation loss : 0.7204012526406183
Validation accuracy : 56.0
Epoch(39/50)
Training loss : 0.3802430905401707
Training accuracy : 94.1
Validation loss : 0.7377102394898732
Validation accuracy : 54.266666666666666
Epoch(40/50)
Training loss : 0.35363613069057465
Training accuracy : 97.425
Validation loss : 0.7262022640970018
Validation accuracy : 55.44444444444444
Epoch(41/50)
Training loss : 0.3645468935370445
Training accuracy : 96.025
Validation loss : 0.7301969660653008
Validation accuracy : 53.8
Epoch(42/50)
Training loss : 0.3560195297002792
Training accuracy : 97.1
Validation loss : 0.7337199919753604
Validation accuracy : 54.355555555555554
Epoch(43/50)
Training loss : 0.352166748046875
Training accuracy : 97.8
Validation loss : 0.7452851679590013
Validation accuracy : 52.355555555555554
Epoch(44/50)
Training loss : 0.35464668050408366
Training accuracy : 96.6
Validation loss : 0.7202744868066576
Validation accuracy : 54.91111111111111
Epoch(45/50)
Training loss : 0.3460316017270088
Training accuracy : 97.8
Validation loss : 0.7232877373695373
Validation accuracy : 56.17777777777778
Epoch(46/50)
Training loss : 0.33223793283104897
Training accuracy : 98.5
Validation loss : 0.7294921106762356
Validation accuracy : 54.333333333333336
Epoch(47/50)
Training loss : 0.3564108729362488
Training accuracy : 96.85
Validation loss : 0.7203237348132663
Validation accuracy : 55.17777777777778
Epoch(48/50)
Training loss : 0.35711230561137197
Training accuracy : 96.55
Validation loss : 0.7195617079734802
Validation accuracy : 56.55555555555556
Epoch(49/50)
Training loss : 0.34381301552057264
Training accuracy : 97.9
Validation loss : 0.7243633601400588
Validation accuracy : 56.022222222222226

#plotting training and validation loss and accuracy
train_val_acc_df = pd.DataFrame.from_dict(accuracy_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})
train_val_loss_df = pd.DataFrame.from_dict(loss_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})

fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(30,10))

sns.lineplot(data=train_val_acc_df, x = "epochs", y="value", hue="variable", ax = axes[0]).set_title('Accuracy/Epoch without dropout')
sns.lineplot(data=train_val_loss_df, x = "epochs", y="value", hue="variable", ax = axes[1]).set_title('Loss/Epoch without dropout')
plt.show()

#Testing out model
y_pred_list = []
y_true_list = []
model.eval()
with torch.no_grad():
    num_correct = 0
    total = 0
    #set_trace()
    for batch, (images, labels) in enumerate(testloader,1):
        pred = model(images)
        pred = torch.argmax(pred, 1)
        y_pred_list.append(pred.numpy())
        y_true_list.append(labels.numpy())
        total += labels.size(0)
        num_correct += (pred == labels).sum().item()
        print(f'Batch ({batch}/{len(testloader)})')
        
        #if batch == 7:
            #break

    print(f'Accuracy of the model on {total} test images: {num_correct * 100 / total}% ')

Batch (1/25)
Batch (2/25)
Batch (3/25)
Batch (4/25)
Batch (5/25)
Batch (6/25)
Batch (7/25)
Batch (8/25)
Batch (9/25)
Batch (10/25)
Batch (11/25)
Batch (12/25)
Batch (13/25)
Batch (14/25)
Batch (15/25)
Batch (16/25)
Batch (17/25)
Batch (18/25)
Batch (19/25)
Batch (20/25)
Batch (21/25)
Batch (22/25)
Batch (23/25)
Batch (24/25)
Batch (25/25)
Accuracy of the model on 781 test images: 56.97823303457106% 

#Recording our experiment
exp_name = f"Baseline Neural Network without Augmentation, Regularization, Batchnorm and Dropout"
expLog.loc[0,:4] = [f"{exp_name}"] + list(np.round(
               [accuracy_stats['train'][-1], 
                accuracy_stats['val'][-1],
                (num_correct * 100 / total)],3))
expLog

/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py:719: FutureWarning: Slicing a positional slice with .loc is not supported, and will raise TypeError in a future version.  Use .loc with labels or .iloc with positions instead.
  indexer = self._get_setitem_indexer(key)

	exp_name	Train Acc	Valid Acc	Test Acc
0	Baseline Neural Network without Augmentation, ...	97.9	56.022	56.978

#calculating the absolute number of right and wrong classifcation to help us 
#get the classification report and confusion matrix
y_pred_list = []
y_true_list = []
with torch.no_grad():
    for batch, (images, labels) in enumerate(testloader,1):      
        y_test_pred = model(images)
        _, y_pred_tag = torch.max(y_test_pred, dim = 1)        
        y_pred_list = [*y_pred_list,*y_pred_tag.cpu().numpy()]
        y_true_list = [*y_true_list,*labels.cpu().numpy()]

y_pred_list_1 = np.array(y_pred_list).T
y_true_list_1 = np.array(y_true_list).T
#y_pred_list_1

print(classification_report(y_true_list_1, y_pred_list_1))

              precision    recall  f1-score   support

           0       0.58      0.57      0.58       402
           1       0.55      0.57      0.56       379

    accuracy                           0.57       781
   macro avg       0.57      0.57      0.57       781
weighted avg       0.57      0.57      0.57       781

print(confusion_matrix(y_true_list_1, y_pred_list_1))

[[228 174]
 [162 217]]

#Plotting the confusion matrix
idx2class = {v: k for k, v in train_it.class_to_idx.items()}
confusion_matrix_df = pd.DataFrame(confusion_matrix(y_true_list_1, y_pred_list_1)).rename(columns=idx2class, index=idx2class)
fig, ax = plt.subplots(figsize=(7,5))         
sns.heatmap(confusion_matrix_df, annot=True, ax=ax, cmap = 'Blues', fmt = 'g')
plt.show()

With Regularization and Dropout

from torch.optim import Adam
class cadod(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(128*128*3, 128)
        #self.fc2 = nn.Linear(1024,512)
        #self.fc3 = nn.Linear(512,128)
        self.fc4 = nn.Linear(128,64)
        self.fc5 = nn.Linear(64,2)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(0.2)
        #self.norm1 = nn.BatchNorm1d(1024)
        self.norm2 = nn.BatchNorm1d(512)
        self.norm3 = nn.BatchNorm1d(128)
        self.norm4 = nn.BatchNorm1d(64)


    def forward(self, x):
        x = x.view(x.shape[0], -1)
        x = F.relu(self.fc1(x))
        #x = self.norm1(x)
        #x = F.relu(self.fc2(x))
        #x = self.norm2(x)
        #x = F.relu(self.fc3(x))
        x = self.norm3(x)
        x = self.dropout(x)
        x = F.relu(self.fc4(x))
        x = self.norm4(x)
        x = self.dropout(x)
        x = self.fc5(x) 
        x = F.softmax(x, dim = 1)
        return x

model = cadod()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.0002, weight_decay = 3e-3)
scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones = [500,1000,1500], gamma = 0.5)

accuracy_stats = {
    'train': [],
    "val": []
}
loss_stats = {
    'train': [],
    "val": []
}

model = model.to(device)
#optimizer = Adam(filter(lambda p: p.requires_grad, model.parameters()))

num_epochs = 50


for e in range(num_epochs):
    cum_epoch_loss = 0
    cum_acc = 0
    batch_loss = 0
    model.train()
    for batch, (images, labels) in enumerate(trainloader,1):
    #print(labels)
    #print(images.shape)
        images = images.to(device)
        labels = labels.to(device)
        optimizer.zero_grad()
        label_pred = model(images).squeeze()
        #print(logps)
        #break;
        #labels = labels.unsqueeze(1)
        loss = criterion(label_pred, labels)
        acc = binary_acc(label_pred, labels)
        loss.backward()
        optimizer.step()
        batch_loss += loss.item()
        cum_acc += acc.item()
        scheduler.step()
        #print(f'Epoch({e}/{num_epochs} : Batch number({batch}/{len(trainloader)})')
    
    with torch.no_grad():
        model.eval()
        val_epoch_loss = 0
        val_epoch_acc = 0
        for batch, (X_val_batch, y_val_batch) in enumerate(valloader,1):
            X_val_batch, y_val_batch = X_val_batch.to(device), y_val_batch.to(device)            
            y_val_pred = model(X_val_batch).squeeze()
            #y_val_pred = torch.unsqueeze(y_val_pred, 0)            
            val_loss = criterion(y_val_pred, y_val_batch)
            val_acc = binary_acc(y_val_pred, y_val_batch)            
            val_epoch_loss += val_loss.item()
            val_epoch_acc += val_acc.item()
        #print(f'Epoch({e}/{num_epochs} : Train loss : {loss.item()} : Train Accuracy : {acc.item()} : Valid loss : {val_loss.item()} : Valid Accuracy : {val_acc.item()}')
    
    loss_stats['train'].append(batch_loss/len(trainloader))
    loss_stats['val'].append(val_epoch_loss/len(valloader))
    accuracy_stats['train'].append(cum_acc/len(trainloader))
    accuracy_stats['val'].append(val_epoch_acc/len(valloader))
    
    print(f'Epoch({e}/{num_epochs})')
    print(f'Training loss : {batch_loss/len(trainloader)}')  
    print(f'Training accuracy : {cum_acc/len(trainloader)}')  
    print(f'Validation loss : {val_epoch_loss/len(valloader)}')  
    print(f'Validation accuracy : {val_epoch_acc/len(valloader)}') 
    

Epoch(0/50)
Training loss : 0.7061507776379585
Training accuracy : 51.375
Validation loss : 0.6786177966329786
Validation accuracy : 56.577777777777776
Epoch(1/50)
Training loss : 0.6667854681611061
Training accuracy : 58.9
Validation loss : 0.6765707095464071
Validation accuracy : 58.6
Epoch(2/50)
Training loss : 0.6524740591645241
Training accuracy : 61.575
Validation loss : 0.6764273669984605
Validation accuracy : 56.15555555555556
Epoch(3/50)
Training loss : 0.6436463922262192
Training accuracy : 63.875
Validation loss : 0.6758760372797649
Validation accuracy : 58.733333333333334
Epoch(4/50)
Training loss : 0.623135557770729
Training accuracy : 67.05
Validation loss : 0.6760422998004489
Validation accuracy : 56.77777777777778
Epoch(5/50)
Training loss : 0.6147623524069786
Training accuracy : 68.95
Validation loss : 0.6819179680612352
Validation accuracy : 55.48888888888889
Epoch(6/50)
Training loss : 0.5993827551603317
Training accuracy : 71.15
Validation loss : 0.6770801941553751
Validation accuracy : 55.06666666666667
Epoch(7/50)
Training loss : 0.573562178015709
Training accuracy : 74.975
Validation loss : 0.6852782156732348
Validation accuracy : 55.8
Epoch(8/50)
Training loss : 0.5674595631659031
Training accuracy : 75.225
Validation loss : 0.6800512472788492
Validation accuracy : 56.17777777777778
Epoch(9/50)
Training loss : 0.5555338352918625
Training accuracy : 76.425
Validation loss : 0.6770689725875855
Validation accuracy : 57.6
Epoch(10/50)
Training loss : 0.5410948015749455
Training accuracy : 78.175
Validation loss : 0.6843940099080403
Validation accuracy : 56.62222222222222
Epoch(11/50)
Training loss : 0.5400927238166332
Training accuracy : 78.0
Validation loss : 0.6958229541778564
Validation accuracy : 55.82222222222222
Epoch(12/50)
Training loss : 0.5191808275878429
Training accuracy : 81.75
Validation loss : 0.6893815000851949
Validation accuracy : 55.84444444444444
Epoch(13/50)
Training loss : 0.48867675811052325
Training accuracy : 85.15
Validation loss : 0.6846356603834364
Validation accuracy : 55.93333333333333
Epoch(14/50)
Training loss : 0.48602257668972015
Training accuracy : 85.675
Validation loss : 0.6952739397684733
Validation accuracy : 55.044444444444444
Epoch(15/50)
Training loss : 0.4678441733121872
Training accuracy : 87.05
Validation loss : 0.6914381715986464
Validation accuracy : 56.53333333333333
Epoch(16/50)
Training loss : 0.4627810247242451
Training accuracy : 88.2
Validation loss : 0.6862888521618313
Validation accuracy : 56.666666666666664
Epoch(17/50)
Training loss : 0.4530599869787693
Training accuracy : 88.1
Validation loss : 0.6942965977721745
Validation accuracy : 55.644444444444446
Epoch(18/50)
Training loss : 0.45015865564346313
Training accuracy : 88.625
Validation loss : 0.6910724798838298
Validation accuracy : 57.13333333333333
Epoch(19/50)
Training loss : 0.4623967081308365
Training accuracy : 87.35
Validation loss : 0.6919400248262617
Validation accuracy : 57.111111111111114
Epoch(20/50)
Training loss : 0.452835438400507
Training accuracy : 87.925
Validation loss : 0.6976125041643778
Validation accuracy : 57.266666666666666
Epoch(21/50)
Training loss : 0.4433533102273941
Training accuracy : 88.725
Validation loss : 0.6950299249755012
Validation accuracy : 57.111111111111114
Epoch(22/50)
Training loss : 0.43154594525694845
Training accuracy : 89.925
Validation loss : 0.68526627752516
Validation accuracy : 58.53333333333333
Epoch(23/50)
Training loss : 0.4356394700706005
Training accuracy : 89.575
Validation loss : 0.7007666958702935
Validation accuracy : 56.733333333333334
Epoch(24/50)
Training loss : 0.4194630056619644
Training accuracy : 90.95
Validation loss : 0.6983552734057109
Validation accuracy : 56.333333333333336
Epoch(25/50)
Training loss : 0.4089778453111649
Training accuracy : 92.525
Validation loss : 0.7079209764798482
Validation accuracy : 54.24444444444445
Epoch(26/50)
Training loss : 0.4014054037630558
Training accuracy : 93.175
Validation loss : 0.6975623852676816
Validation accuracy : 57.422222222222224
Epoch(27/50)
Training loss : 0.3958094328641891
Training accuracy : 93.875
Validation loss : 0.7030816886160108
Validation accuracy : 57.13333333333333
Epoch(28/50)
Training loss : 0.38626367375254633
Training accuracy : 94.45
Validation loss : 0.7000549064742194
Validation accuracy : 56.46666666666667
Epoch(29/50)
Training loss : 0.38705129474401473
Training accuracy : 94.35
Validation loss : 0.6993470927079518
Validation accuracy : 56.84444444444444
Epoch(30/50)
Training loss : 0.39100720062851907
Training accuracy : 94.175
Validation loss : 0.7082764228185018
Validation accuracy : 55.31111111111111
Epoch(31/50)
Training loss : 0.40191668942570685
Training accuracy : 93.0
Validation loss : 0.6901014036602444
Validation accuracy : 58.22222222222222
Epoch(32/50)
Training loss : 0.39481541961431504
Training accuracy : 94.325
Validation loss : 0.7064493921067979
Validation accuracy : 55.68888888888889
Epoch(33/50)
Training loss : 0.38618241250514984
Training accuracy : 95.175
Validation loss : 0.696589375866784
Validation accuracy : 57.53333333333333
Epoch(34/50)
Training loss : 0.3873636096715927
Training accuracy : 94.775
Validation loss : 0.6966616405381096
Validation accuracy : 57.644444444444446
Epoch(35/50)
Training loss : 0.38365463837981223
Training accuracy : 94.625
Validation loss : 0.6949302713076274
Validation accuracy : 59.08888888888889
Epoch(36/50)
Training loss : 0.3800564780831337
Training accuracy : 95.225
Validation loss : 0.6917685647805532
Validation accuracy : 59.044444444444444
Epoch(37/50)
Training loss : 0.37661325857043265
Training accuracy : 95.65
Validation loss : 0.6992703444427915
Validation accuracy : 57.06666666666667
Epoch(38/50)
Training loss : 0.37502980008721354
Training accuracy : 95.65
Validation loss : 0.70349523557557
Validation accuracy : 56.91111111111111
Epoch(39/50)
Training loss : 0.372482930123806
Training accuracy : 95.7
Validation loss : 0.6991914086871677
Validation accuracy : 56.53333333333333
Epoch(40/50)
Training loss : 0.36446407809853554
Training accuracy : 96.65
Validation loss : 0.7055752979384529
Validation accuracy : 56.022222222222226
Epoch(41/50)
Training loss : 0.3738012857735157
Training accuracy : 95.875
Validation loss : 0.7058716734250386
Validation accuracy : 56.2
Epoch(42/50)
Training loss : 0.36233029812574385
Training accuracy : 96.725
Validation loss : 0.7093054506513807
Validation accuracy : 55.577777777777776
Epoch(43/50)
Training loss : 0.3614585630595684
Training accuracy : 96.75
Validation loss : 0.7083631482389238
Validation accuracy : 55.86666666666667
Epoch(44/50)
Training loss : 0.3596478894352913
Training accuracy : 96.95
Validation loss : 0.6960457702477773
Validation accuracy : 57.13333333333333
Epoch(45/50)
Training loss : 0.3619891867041588
Training accuracy : 96.45
Validation loss : 0.7024208267529806
Validation accuracy : 56.82222222222222
Epoch(46/50)
Training loss : 0.3646397091448307
Training accuracy : 96.25
Validation loss : 0.7022896779908074
Validation accuracy : 56.24444444444445
Epoch(47/50)
Training loss : 0.3578439809381962
Training accuracy : 97.225
Validation loss : 0.7054678320884704
Validation accuracy : 55.955555555555556
Epoch(48/50)
Training loss : 0.37112242430448533
Training accuracy : 96.15
Validation loss : 0.7119460112518734
Validation accuracy : 55.62222222222222
Epoch(49/50)
Training loss : 0.36486905068159103
Training accuracy : 96.325
Validation loss : 0.7079982015821669
Validation accuracy : 56.46666666666667

import matplotlib.pyplot as plt
import seaborn as sns
train_val_acc_df = pd.DataFrame.from_dict(accuracy_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})
train_val_loss_df = pd.DataFrame.from_dict(loss_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})

fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(30,10))

sns.lineplot(data=train_val_acc_df, x = "epochs", y="value", hue="variable", ax = axes[0]).set_title('Accuracy/Epoch')
sns.lineplot(data=train_val_loss_df, x = "epochs", y="value", hue="variable", ax = axes[1]).set_title('Loss/Epoch')
plt.show()

y_pred_list = []
y_true_list = []
model.eval()
with torch.no_grad():
    num_correct = 0
    total = 0
    #set_trace()
    for batch, (images, labels) in enumerate(testloader,1):
        pred = model(images)
        #output = torch.exp(logps)
        pred = torch.argmax(pred, 1)
        y_pred_list.append(pred.numpy())
        y_true_list.append(labels.numpy())
        total += labels.size(0)
        num_correct += (pred == labels).sum().item()
        #print('accuracy', binary_acc(pred, labels).item())
        print(f'Batch ({batch}/{len(testloader)})')
        
        #if batch == 7:
            #break

    print(f'Accuracy of the model on {total} test images: {num_correct * 100 / total}% ')

Batch (1/25)
Batch (2/25)
Batch (3/25)
Batch (4/25)
Batch (5/25)
Batch (6/25)
Batch (7/25)
Batch (8/25)
Batch (9/25)
Batch (10/25)
Batch (11/25)
Batch (12/25)
Batch (13/25)
Batch (14/25)
Batch (15/25)
Batch (16/25)
Batch (17/25)
Batch (18/25)
Batch (19/25)
Batch (20/25)
Batch (21/25)
Batch (22/25)
Batch (23/25)
Batch (24/25)
Batch (25/25)
Accuracy of the model on 781 test images: 57.10627400768246% 

exp_name = f"Neural Network without Augmentation and with Regularization and Dropout"
expLog.loc[1,:4] = [f"{exp_name}"] + list(np.round(
               [accuracy_stats['train'][-1], 
                accuracy_stats['val'][-1],
                (num_correct * 100 / total)],3))
expLog

/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py:719: FutureWarning: Slicing a positional slice with .loc is not supported, and will raise TypeError in a future version.  Use .loc with labels or .iloc with positions instead.
  indexer = self._get_setitem_indexer(key)

	exp_name	Train Acc	Valid Acc	Test Acc
0	Baseline Neural Network without Augmentation, ...	97.9	56.022	56.978
1	Neural Network without Augmentation and with R...	96.325	56.467	57.106

y_pred_list = []
y_true_list = []
with torch.no_grad():
    for batch, (images, labels) in enumerate(testloader,1):      
        y_test_pred = model(images)
        _, y_pred_tag = torch.max(y_test_pred, dim = 1)        
        y_pred_list = [*y_pred_list,*y_pred_tag.cpu().numpy()]
        y_true_list = [*y_true_list,*labels.cpu().numpy()]

y_pred_list_1 = np.array(y_pred_list).T
y_true_list_1 = np.array(y_true_list).T
#y_pred_list_1

print(classification_report(y_true_list_1, y_pred_list_1))

              precision    recall  f1-score   support

           0       0.58      0.63      0.60       402
           1       0.56      0.51      0.53       379

    accuracy                           0.57       781
   macro avg       0.57      0.57      0.57       781
weighted avg       0.57      0.57      0.57       781

print(confusion_matrix(y_true_list_1, y_pred_list_1))

[[254 148]
 [187 192]]

idx2class = {v: k for k, v in train_it.class_to_idx.items()}
confusion_matrix_df = pd.DataFrame(confusion_matrix(y_true_list_1, y_pred_list_1)).rename(columns=idx2class, index=idx2class)
fig, ax = plt.subplots(figsize=(7,5))         
sns.heatmap(confusion_matrix_df, annot=True, ax=ax, cmap = 'Blues', fmt = 'g')
plt.show()

Data Augmentation

#Data augmentation
mean = [0.485, 0.456, 0.406]
std = [0.229, 0.224, 0.225]

transform_test = transforms.Compose([
    #transforms.ToPILImage(),             
    transforms.Resize((128, 128)),
    transforms.ToTensor(),
    transforms.Normalize(mean=mean, std=std)
    ])
transform_train = transforms.Compose([
    #transforms.ToPILImage(),             
    transforms.Resize((128, 128)),
    transforms.RandomHorizontalFlip(),
    transforms.RandomRotation(40),
    transforms.ToTensor(),
    transforms.Normalize(mean=mean, std=std)
    #transforms.RandomAutocontrast()
    
    ])

train_it = datasets.ImageFolder('cat_vs_dog/train/', transform=transform_train)
test_it = datasets.ImageFolder('cat_vs_dog/test/', transform=transform_test)

dataset_size = len(train_it)
dataset_indices = list(range(dataset_size))
np.random.shuffle(dataset_indices)

dataset_size

3219

train_set, val_set = torch.utils.data.random_split(train_it, [2500, 719]) 

trainloader = DataLoader(train_set, batch_size=54, shuffle=True)
valloader = DataLoader(val_set, shuffle=True, batch_size=16)
testloader = DataLoader(test_it, batch_size=16, shuffle=True)

for images, labels in trainloader:
    print(images.size(), labels.size())
    print(labels)
    break

torch.Size([54, 3, 128, 128]) torch.Size([54])
tensor([0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1,
        1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 1,
        0, 1, 0, 0, 1, 0])

for batch, (images, labels) in enumerate(trainloader,1):
    print(sum(labels))
    break

tensor(30)

With Data Augmentation

Without regularization and dropout

from torch.optim import Adam
class cadod(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(128*128*3, 128)
        #self.fc2 = nn.Linear(1024,512)
        #self.fc3 = nn.Linear(512,128)
        self.fc4 = nn.Linear(128,64)
        self.fc5 = nn.Linear(64,2)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(0.2)
        #self.norm1 = nn.BatchNorm1d(1024)
        self.norm2 = nn.BatchNorm1d(512)
        self.norm3 = nn.BatchNorm1d(128)
        self.norm4 = nn.BatchNorm1d(64)


    def forward(self, x):
        x = x.view(x.shape[0], -1)
        x = F.relu(self.fc1(x))

        x = self.norm3(x)
        x = F.relu(self.fc4(x))
        x = self.norm4(x)
        #x =self.dropout(x)
        x = self.fc5(x) 
        x = F.softmax(x, dim = 1)
        return x

model = cadod()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.0002)
#scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones = [500,1000,1500], gamma = 0.5)

accuracy_stats = {
    'train': [],
    "val": []
}
loss_stats = {
    'train': [],
    "val": []
}

model = model.to(device)
#optimizer = Adam(filter(lambda p: p.requires_grad, model.parameters()))

num_epochs = 50


for e in range(num_epochs):
    cum_epoch_loss = 0
    cum_acc = 0
    batch_loss = 0
    model.train()
    for batch, (images, labels) in enumerate(trainloader,1):
    #print(labels)
    #print(images.shape)
        images = images.to(device)
        labels = labels.to(device)
        optimizer.zero_grad()
        label_pred = model(images).squeeze()
        #print(logps)
        #break;
        #labels = labels.unsqueeze(1)
        loss = criterion(label_pred, labels)
        acc = binary_acc(label_pred, labels)
        loss.backward()
        optimizer.step()
        batch_loss += loss.item()
        cum_acc += acc.item()
        #scheduler.step()
        #print(f'Epoch({e}/{num_epochs} : Batch number({batch}/{len(trainloader)})')
    
    with torch.no_grad():
        model.eval()
        val_epoch_loss = 0
        val_epoch_acc = 0
        for batch, (X_val_batch, y_val_batch) in enumerate(valloader,1):
            X_val_batch, y_val_batch = X_val_batch.to(device), y_val_batch.to(device)            
            y_val_pred = model(X_val_batch).squeeze()
            #y_val_pred = torch.unsqueeze(y_val_pred, 0)            
            val_loss = criterion(y_val_pred, y_val_batch)
            val_acc = binary_acc(y_val_pred, y_val_batch)            
            val_epoch_loss += val_loss.item()
            val_epoch_acc += val_acc.item()
        #print(f'Epoch({e}/{num_epochs} : Train loss : {loss.item()} : Train Accuracy : {acc.item()} : Valid loss : {val_loss.item()} : Valid Accuracy : {val_acc.item()}')
    
    loss_stats['train'].append(batch_loss/len(trainloader))
    loss_stats['val'].append(val_epoch_loss/len(valloader))
    accuracy_stats['train'].append(cum_acc/len(trainloader))
    accuracy_stats['val'].append(val_epoch_acc/len(valloader))
    
    print(f'Epoch({e}/{num_epochs})')
    print(f'Training loss : {batch_loss/len(trainloader)}')  
    print(f'Training accuracy : {cum_acc/len(trainloader)}')  
    print(f'Validation loss : {val_epoch_loss/len(valloader)}')  
    print(f'Validation accuracy : {val_epoch_acc/len(valloader)}') 
    

Epoch(0/50)
Training loss : 0.69586806474848
Training accuracy : 54.08510638297872
Validation loss : 0.6855854153633117
Validation accuracy : 56.86666666666667
Epoch(1/50)
Training loss : 0.6864400341155681
Training accuracy : 56.297872340425535
Validation loss : 0.6852175182766385
Validation accuracy : 53.8
Epoch(2/50)
Training loss : 0.6826955536578564
Training accuracy : 55.93617021276596
Validation loss : 0.6779671774970161
Validation accuracy : 57.266666666666666
Epoch(3/50)
Training loss : 0.6778146459701213
Training accuracy : 56.61702127659574
Validation loss : 0.6752467685275607
Validation accuracy : 55.888888888888886
Epoch(4/50)
Training loss : 0.6754652134915615
Training accuracy : 57.787234042553195
Validation loss : 0.6792816585964627
Validation accuracy : 55.75555555555555
Epoch(5/50)
Training loss : 0.6676017324975196
Training accuracy : 59.04255319148936
Validation loss : 0.6839485155211554
Validation accuracy : 53.91111111111111
Epoch(6/50)
Training loss : 0.6690755575261218
Training accuracy : 59.659574468085104
Validation loss : 0.682853737142351
Validation accuracy : 54.4
Epoch(7/50)
Training loss : 0.665417713053683
Training accuracy : 58.212765957446805
Validation loss : 0.6688671562406752
Validation accuracy : 58.111111111111114
Epoch(8/50)
Training loss : 0.6630925143018682
Training accuracy : 60.723404255319146
Validation loss : 0.6752253505918715
Validation accuracy : 57.355555555555554
Epoch(9/50)
Training loss : 0.6621135587387896
Training accuracy : 60.702127659574465
Validation loss : 0.6806839479340447
Validation accuracy : 58.044444444444444
Epoch(10/50)
Training loss : 0.6623214381806394
Training accuracy : 60.765957446808514
Validation loss : 0.6798261761665344
Validation accuracy : 58.46666666666667
Epoch(11/50)
Training loss : 0.6594443929956314
Training accuracy : 60.808510638297875
Validation loss : 0.6691505445374383
Validation accuracy : 59.22222222222222
Epoch(12/50)
Training loss : 0.652525356475343
Training accuracy : 61.59574468085106
Validation loss : 0.6731579555405511
Validation accuracy : 59.15555555555556
Epoch(13/50)
Training loss : 0.6545437549022918
Training accuracy : 62.1063829787234
Validation loss : 0.6726664490169949
Validation accuracy : 57.93333333333333
Epoch(14/50)
Training loss : 0.6485119905877621
Training accuracy : 62.276595744680854
Validation loss : 0.6796364982922872
Validation accuracy : 56.955555555555556
Epoch(15/50)
Training loss : 0.6515876168900347
Training accuracy : 61.702127659574465
Validation loss : 0.6812245845794678
Validation accuracy : 57.955555555555556
Epoch(16/50)
Training loss : 0.6524163446527846
Training accuracy : 62.42553191489362
Validation loss : 0.6739172048038906
Validation accuracy : 56.75555555555555
Epoch(17/50)
Training loss : 0.6496407237458737
Training accuracy : 61.57446808510638
Validation loss : 0.6786000490188598
Validation accuracy : 56.93333333333333
Epoch(18/50)
Training loss : 0.6486472855222986
Training accuracy : 62.4468085106383
Validation loss : 0.6786557078361511
Validation accuracy : 58.75555555555555
Epoch(19/50)
Training loss : 0.6498188883700269
Training accuracy : 62.91489361702128
Validation loss : 0.6626882327927484
Validation accuracy : 58.77777777777778
Epoch(20/50)
Training loss : 0.6441015804067571
Training accuracy : 63.57446808510638
Validation loss : 0.665606168905894
Validation accuracy : 60.15555555555556
Epoch(21/50)
Training loss : 0.643499770063035
Training accuracy : 64.17021276595744
Validation loss : 0.6724937942292956
Validation accuracy : 58.93333333333333
Epoch(22/50)
Training loss : 0.6442216330386222
Training accuracy : 63.234042553191486
Validation loss : 0.6755911919805738
Validation accuracy : 57.28888888888889
Epoch(23/50)
Training loss : 0.6406150472925064
Training accuracy : 63.95744680851064
Validation loss : 0.6829689330524868
Validation accuracy : 55.55555555555556
Epoch(24/50)
Training loss : 0.6335100950078761
Training accuracy : 65.87234042553192
Validation loss : 0.6714744475152757
Validation accuracy : 57.955555555555556
Epoch(25/50)
Training loss : 0.6381233689632821
Training accuracy : 64.93617021276596
Validation loss : 0.6745229151513842
Validation accuracy : 56.266666666666666
Epoch(26/50)
Training loss : 0.6368751538560745
Training accuracy : 65.61702127659575
Validation loss : 0.6846093985769484
Validation accuracy : 57.08888888888889
Epoch(27/50)
Training loss : 0.6377042988513378
Training accuracy : 65.14893617021276
Validation loss : 0.6666999724176195
Validation accuracy : 59.044444444444444
Epoch(28/50)
Training loss : 0.625789483810993
Training accuracy : 67.61702127659575
Validation loss : 0.6683956212467618
Validation accuracy : 57.68888888888889
Epoch(29/50)
Training loss : 0.6385705711993765
Training accuracy : 63.93617021276596
Validation loss : 0.6656111664242215
Validation accuracy : 59.022222222222226
Epoch(30/50)
Training loss : 0.6292796794404375
Training accuracy : 65.74468085106383
Validation loss : 0.6950658758481344
Validation accuracy : 56.0
Epoch(31/50)
Training loss : 0.6275236467097668
Training accuracy : 66.2127659574468
Validation loss : 0.6720664262771606
Validation accuracy : 59.28888888888889
Epoch(32/50)
Training loss : 0.6251293433473465
Training accuracy : 65.1063829787234
Validation loss : 0.6714218378067016
Validation accuracy : 56.93333333333333
Epoch(33/50)
Training loss : 0.6208612234034436
Training accuracy : 67.42553191489361
Validation loss : 0.6812989168696933
Validation accuracy : 56.48888888888889
Epoch(34/50)
Training loss : 0.6172107001568409
Training accuracy : 67.91489361702128
Validation loss : 0.6701238552729288
Validation accuracy : 57.22222222222222
Epoch(35/50)
Training loss : 0.6218452250703852
Training accuracy : 66.93617021276596
Validation loss : 0.6786299334632026
Validation accuracy : 56.71111111111111
Epoch(36/50)
Training loss : 0.61522428152409
Training accuracy : 68.04255319148936
Validation loss : 0.6827057851685419
Validation accuracy : 57.53333333333333
Epoch(37/50)
Training loss : 0.6170673459134204
Training accuracy : 67.42553191489361
Validation loss : 0.6858004185888502
Validation accuracy : 58.06666666666667
Epoch(38/50)
Training loss : 0.6230450107696208
Training accuracy : 66.80851063829788
Validation loss : 0.6620791739887661
Validation accuracy : 59.6
Epoch(39/50)
Training loss : 0.6139012283467232
Training accuracy : 68.1063829787234
Validation loss : 0.6844247877597809
Validation accuracy : 56.955555555555556
Epoch(40/50)
Training loss : 0.6146208542458554
Training accuracy : 68.14893617021276
Validation loss : 0.684525113635593
Validation accuracy : 57.333333333333336
Epoch(41/50)
Training loss : 0.6127230005061373
Training accuracy : 67.76595744680851
Validation loss : 0.6782424807548523
Validation accuracy : 58.44444444444444
Epoch(42/50)
Training loss : 0.6190533473136577
Training accuracy : 66.80851063829788
Validation loss : 0.6791880541377597
Validation accuracy : 56.4
Epoch(43/50)
Training loss : 0.6100762854231164
Training accuracy : 68.65957446808511
Validation loss : 0.6733865035904778
Validation accuracy : 58.022222222222226
Epoch(44/50)
Training loss : 0.6073861565995724
Training accuracy : 69.08510638297872
Validation loss : 0.6692661060227288
Validation accuracy : 60.91111111111111
Epoch(45/50)
Training loss : 0.5976200839306446
Training accuracy : 71.04255319148936
Validation loss : 0.6703213122155931
Validation accuracy : 58.53333333333333
Epoch(46/50)
Training loss : 0.5934598242982905
Training accuracy : 71.59574468085107
Validation loss : 0.6804684460163116
Validation accuracy : 57.888888888888886
Epoch(47/50)
Training loss : 0.5934890191605751
Training accuracy : 71.80851063829788
Validation loss : 0.6780909279982249
Validation accuracy : 57.71111111111111
Epoch(48/50)
Training loss : 0.5968403207494858
Training accuracy : 69.76595744680851
Validation loss : 0.6849978725115459
Validation accuracy : 56.266666666666666
Epoch(49/50)
Training loss : 0.5946247336712289
Training accuracy : 70.70212765957447
Validation loss : 0.6861440870496962
Validation accuracy : 57.51111111111111

import matplotlib.pyplot as plt
import seaborn as sns
train_val_acc_df = pd.DataFrame.from_dict(accuracy_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})
train_val_loss_df = pd.DataFrame.from_dict(loss_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})

fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(30,10))

sns.lineplot(data=train_val_acc_df, x = "epochs", y="value", hue="variable", ax = axes[0]).set_title('Accuracy/Epoch without dropout')
sns.lineplot(data=train_val_loss_df, x = "epochs", y="value", hue="variable", ax = axes[1]).set_title('Loss/Epoch without dropout')
plt.show()

y_pred_list = []
y_true_list = []
model.eval()
with torch.no_grad():
    num_correct = 0
    total = 0
    #set_trace()
    for batch, (images, labels) in enumerate(testloader,1):
        pred = model(images)
        #output = torch.exp(logps)
        pred = torch.argmax(pred, 1)
        y_pred_list.append(pred.numpy())
        y_true_list.append(labels.numpy())
        total += labels.size(0)
        num_correct += (pred == labels).sum().item()
        #print('accuracy', binary_acc(pred, labels).item())
        print(f'Batch ({batch}/{len(testloader)})')
        
        #if batch == 7:
            #break

    print(f'Accuracy of the model on {total} test images: {num_correct * 100 / total}% ')

Batch (1/49)
Batch (2/49)
Batch (3/49)
Batch (4/49)
Batch (5/49)
Batch (6/49)
Batch (7/49)
Batch (8/49)
Batch (9/49)
Batch (10/49)
Batch (11/49)
Batch (12/49)
Batch (13/49)
Batch (14/49)
Batch (15/49)
Batch (16/49)
Batch (17/49)
Batch (18/49)
Batch (19/49)
Batch (20/49)
Batch (21/49)
Batch (22/49)
Batch (23/49)
Batch (24/49)
Batch (25/49)
Batch (26/49)
Batch (27/49)
Batch (28/49)
Batch (29/49)
Batch (30/49)
Batch (31/49)
Batch (32/49)
Batch (33/49)
Batch (34/49)
Batch (35/49)
Batch (36/49)
Batch (37/49)
Batch (38/49)
Batch (39/49)
Batch (40/49)
Batch (41/49)
Batch (42/49)
Batch (43/49)
Batch (44/49)
Batch (45/49)
Batch (46/49)
Batch (47/49)
Batch (48/49)
Batch (49/49)
Accuracy of the model on 781 test images: 59.92317541613316% 

exp_name = f"Neural Network without Regularization and Dropout"
expLog.loc[2,:4] = [f"{exp_name}"] + list(np.round(
               [accuracy_stats['train'][-1], 
                accuracy_stats['val'][-1],
                (num_correct * 100 / total)],3))
expLog

/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py:719: FutureWarning: Slicing a positional slice with .loc is not supported, and will raise TypeError in a future version.  Use .loc with labels or .iloc with positions instead.
  indexer = self._get_setitem_indexer(key)

	exp_name	Train Acc	Valid Acc	Test Acc
0	Baseline Neural Network without Augmentation, ...	97.9	56.022	56.978
1	Neural Network without Augmentation and with R...	96.325	56.467	57.106
2	Neural Network without Regularization and Dropout	70.702	57.511	59.923

y_pred_list = []
y_true_list = []
with torch.no_grad():
    for batch, (images, labels) in enumerate(testloader,1):      
        y_test_pred = model(images)
        _, y_pred_tag = torch.max(y_test_pred, dim = 1)        
        y_pred_list = [*y_pred_list,*y_pred_tag.cpu().numpy()]
        y_true_list = [*y_true_list,*labels.cpu().numpy()]

y_pred_list_1 = np.array(y_pred_list).T
y_true_list_1 = np.array(y_true_list).T
#y_pred_list_1

print(classification_report(y_true_list_1, y_pred_list_1))

              precision    recall  f1-score   support

           0       0.62      0.57      0.59       402
           1       0.58      0.63      0.60       379

    accuracy                           0.60       781
   macro avg       0.60      0.60      0.60       781
weighted avg       0.60      0.60      0.60       781

print(confusion_matrix(y_true_list_1, y_pred_list_1))

[[229 173]
 [140 239]]

idx2class = {v: k for k, v in train_it.class_to_idx.items()}
confusion_matrix_df = pd.DataFrame(confusion_matrix(y_true_list_1, y_pred_list_1)).rename(columns=idx2class, index=idx2class)
fig, ax = plt.subplots(figsize=(7,5))         
sns.heatmap(confusion_matrix_df, annot=True, ax=ax, cmap = 'Blues', fmt = 'g')
plt.show()

With Regularizaion and Without Dropout

from torch.optim import Adam
class cadod(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(128*128*3, 128)
        #self.fc2 = nn.Linear(1024,512)
        #self.fc3 = nn.Linear(512,128)
        self.fc4 = nn.Linear(128,64)
        self.fc5 = nn.Linear(64,2)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(0.2)
        #self.norm1 = nn.BatchNorm1d(1024)
        self.norm2 = nn.BatchNorm1d(512)
        self.norm3 = nn.BatchNorm1d(128)
        self.norm4 = nn.BatchNorm1d(64)


    def forward(self, x):
        x = x.view(x.shape[0], -1)
        x = F.relu(self.fc1(x))
        #x = self.norm1(x)
        #x = F.relu(self.fc2(x))
        #x = self.norm2(x)
        #x = F.relu(self.fc3(x))
        x = self.norm3(x)
        x = F.relu(self.fc4(x))
        x = self.norm4(x)
        #x =self.dropout(x)
        x = self.fc5(x) 
        x = F.softmax(x, dim = 1)
        return x

model = cadod()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.0002, weight_decay = 3e-3)
scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones = [500,1000,1500], gamma = 0.5)

model

cadod(
  (fc1): Linear(in_features=49152, out_features=128, bias=True)
  (fc4): Linear(in_features=128, out_features=64, bias=True)
  (fc5): Linear(in_features=64, out_features=2, bias=True)
  (relu): ReLU()
  (dropout): Dropout(p=0.2, inplace=False)
  (norm2): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (norm3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (norm4): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)

accuracy_stats = {
    'train': [],
    "val": []
}
loss_stats = {
    'train': [],
    "val": []
}

model = model.to(device)
#optimizer = Adam(filter(lambda p: p.requires_grad, model.parameters()))

num_epochs = 50


for e in range(num_epochs):
    cum_epoch_loss = 0
    cum_acc = 0
    batch_loss = 0
    model.train()
    for batch, (images, labels) in enumerate(trainloader,1):
    #print(labels)
    #print(images.shape)
        images = images.to(device)
        labels = labels.to(device)
        optimizer.zero_grad()
        label_pred = model(images).squeeze()
        #print(logps)
        #break;
        #labels = labels.unsqueeze(1)
        loss = criterion(label_pred, labels)
        acc = binary_acc(label_pred, labels)
        loss.backward()
        optimizer.step()
        batch_loss += loss.item()
        cum_acc += acc.item()
        scheduler.step()
        #print(f'Epoch({e}/{num_epochs} : Batch number({batch}/{len(trainloader)})')
    
    with torch.no_grad():
        model.eval()
        val_epoch_loss = 0
        val_epoch_acc = 0
        for batch, (X_val_batch, y_val_batch) in enumerate(valloader,1):
            X_val_batch, y_val_batch = X_val_batch.to(device), y_val_batch.to(device)            
            y_val_pred = model(X_val_batch).squeeze()
            #y_val_pred = torch.unsqueeze(y_val_pred, 0)            
            val_loss = criterion(y_val_pred, y_val_batch)
            val_acc = binary_acc(y_val_pred, y_val_batch)            
            val_epoch_loss += val_loss.item()
            val_epoch_acc += val_acc.item()
        #print(f'Epoch({e}/{num_epochs} : Train loss : {loss.item()} : Train Accuracy : {acc.item()} : Valid loss : {val_loss.item()} : Valid Accuracy : {val_acc.item()}')
    
    loss_stats['train'].append(batch_loss/len(trainloader))
    loss_stats['val'].append(val_epoch_loss/len(valloader))
    accuracy_stats['train'].append(cum_acc/len(trainloader))
    accuracy_stats['val'].append(val_epoch_acc/len(valloader))
    
    print(f'Epoch({e}/{num_epochs})')
    print(f'Training loss : {batch_loss/len(trainloader)}')  
    print(f'Training accuracy : {cum_acc/len(trainloader)}')  
    print(f'Validation loss : {val_epoch_loss/len(valloader)}')  
    print(f'Validation accuracy : {val_epoch_acc/len(valloader)}') 
    

Epoch(0/50)
Training loss : 0.7074646315676101
Training accuracy : 51.53191489361702
Validation loss : 0.6924390620655484
Validation accuracy : 53.75555555555555
Epoch(1/50)
Training loss : 0.6898972113081749
Training accuracy : 55.08510638297872
Validation loss : 0.6938854349984063
Validation accuracy : 54.86666666666667
Epoch(2/50)
Training loss : 0.6837859470793541
Training accuracy : 54.61702127659574
Validation loss : 0.689277368121677
Validation accuracy : 56.17777777777778
Epoch(3/50)
Training loss : 0.6788228521955774
Training accuracy : 57.638297872340424
Validation loss : 0.6829059349166022
Validation accuracy : 55.977777777777774
Epoch(4/50)
Training loss : 0.6816198179062377
Training accuracy : 57.1063829787234
Validation loss : 0.6839883327484131
Validation accuracy : 55.91111111111111
Epoch(5/50)
Training loss : 0.6750124984599174
Training accuracy : 58.297872340425535
Validation loss : 0.6759995142618815
Validation accuracy : 59.06666666666667
Epoch(6/50)
Training loss : 0.6735540539660352
Training accuracy : 57.808510638297875
Validation loss : 0.67955082654953
Validation accuracy : 58.08888888888889
Epoch(7/50)
Training loss : 0.6702524791372583
Training accuracy : 58.97872340425532
Validation loss : 0.6852095815870497
Validation accuracy : 54.955555555555556
Epoch(8/50)
Training loss : 0.6755938796286888
Training accuracy : 57.659574468085104
Validation loss : 0.674956140253279
Validation accuracy : 56.666666666666664
Epoch(9/50)
Training loss : 0.6690128394897948
Training accuracy : 59.808510638297875
Validation loss : 0.6729978905783759
Validation accuracy : 58.022222222222226
Epoch(10/50)
Training loss : 0.6760080735734169
Training accuracy : 57.829787234042556
Validation loss : 0.6731777919663323
Validation accuracy : 57.68888888888889
Epoch(11/50)
Training loss : 0.6606562036149045
Training accuracy : 61.702127659574465
Validation loss : 0.6794908616277907
Validation accuracy : 56.75555555555555
Epoch(12/50)
Training loss : 0.6586204589681423
Training accuracy : 61.59574468085106
Validation loss : 0.6704196943177118
Validation accuracy : 59.4
Epoch(13/50)
Training loss : 0.6653362601361377
Training accuracy : 59.148936170212764
Validation loss : 0.6726820760303074
Validation accuracy : 58.022222222222226
Epoch(14/50)
Training loss : 0.6586187469198349
Training accuracy : 61.148936170212764
Validation loss : 0.673554335700141
Validation accuracy : 58.08888888888889
Epoch(15/50)
Training loss : 0.6573461256128676
Training accuracy : 62.297872340425535
Validation loss : 0.6727242390314738
Validation accuracy : 58.37777777777778
Epoch(16/50)
Training loss : 0.6602142656103094
Training accuracy : 60.829787234042556
Validation loss : 0.6802246411641438
Validation accuracy : 54.733333333333334
Epoch(17/50)
Training loss : 0.6508438764734471
Training accuracy : 62.93617021276596
Validation loss : 0.6734918713569641
Validation accuracy : 58.93333333333333
Epoch(18/50)
Training loss : 0.6584140658378601
Training accuracy : 61.8936170212766
Validation loss : 0.6734908289379544
Validation accuracy : 57.28888888888889
Epoch(19/50)
Training loss : 0.6585366586421398
Training accuracy : 60.808510638297875
Validation loss : 0.6770080473687914
Validation accuracy : 57.44444444444444
Epoch(20/50)
Training loss : 0.6583279561489186
Training accuracy : 60.765957446808514
Validation loss : 0.6727444344096714
Validation accuracy : 59.022222222222226
Epoch(21/50)
Training loss : 0.6522507959223808
Training accuracy : 62.191489361702125
Validation loss : 0.6709768321779039
Validation accuracy : 57.17777777777778
Epoch(22/50)
Training loss : 0.6459899138897023
Training accuracy : 63.02127659574468
Validation loss : 0.6724988235367669
Validation accuracy : 59.62222222222222
Epoch(23/50)
Training loss : 0.6443162281462487
Training accuracy : 62.87234042553192
Validation loss : 0.6774350656403436
Validation accuracy : 59.355555555555554
Epoch(24/50)
Training loss : 0.6430908961498991
Training accuracy : 64.87234042553192
Validation loss : 0.675101085503896
Validation accuracy : 57.06666666666667
Epoch(25/50)
Training loss : 0.6448454374962664
Training accuracy : 63.829787234042556
Validation loss : 0.6723080290688409
Validation accuracy : 59.22222222222222
Epoch(26/50)
Training loss : 0.6381145969350287
Training accuracy : 64.85106382978724
Validation loss : 0.6746341519885592
Validation accuracy : 57.51111111111111
Epoch(27/50)
Training loss : 0.6423022252448062
Training accuracy : 63.93617021276596
Validation loss : 0.6765135606129964
Validation accuracy : 56.55555555555556
Epoch(28/50)
Training loss : 0.6463380291106853
Training accuracy : 63.51063829787234
Validation loss : 0.6716744846767849
Validation accuracy : 58.31111111111111
Epoch(29/50)
Training loss : 0.6400747058239389
Training accuracy : 65.2127659574468
Validation loss : 0.6710040052731832
Validation accuracy : 58.82222222222222
Epoch(30/50)
Training loss : 0.6388176831793277
Training accuracy : 64.36170212765957
Validation loss : 0.6772846062978108
Validation accuracy : 58.48888888888889
Epoch(31/50)
Training loss : 0.6390079272554275
Training accuracy : 65.2127659574468
Validation loss : 0.6778478834364149
Validation accuracy : 57.51111111111111
Epoch(32/50)
Training loss : 0.6320334343200035
Training accuracy : 65.93617021276596
Validation loss : 0.6668278243806627
Validation accuracy : 60.333333333333336
Epoch(33/50)
Training loss : 0.6324139313494905
Training accuracy : 65.31914893617021
Validation loss : 0.673486836751302
Validation accuracy : 59.422222222222224
Epoch(34/50)
Training loss : 0.6264437348284619
Training accuracy : 67.65957446808511
Validation loss : 0.6712869816356235
Validation accuracy : 59.71111111111111
Epoch(35/50)
Training loss : 0.6290726966046273
Training accuracy : 66.19148936170212
Validation loss : 0.6703696105215284
Validation accuracy : 58.84444444444444
Epoch(36/50)
Training loss : 0.6262789685675438
Training accuracy : 67.74468085106383
Validation loss : 0.6660648716820611
Validation accuracy : 60.93333333333333
Epoch(37/50)
Training loss : 0.6236991286277771
Training accuracy : 67.31914893617021
Validation loss : 0.6709921744134691
Validation accuracy : 58.22222222222222
Epoch(38/50)
Training loss : 0.6224406589853003
Training accuracy : 68.02127659574468
Validation loss : 0.6766502843962775
Validation accuracy : 57.644444444444446
Epoch(39/50)
Training loss : 0.6193873856930022
Training accuracy : 68.17021276595744
Validation loss : 0.6813429262903001
Validation accuracy : 57.28888888888889
Epoch(40/50)
Training loss : 0.6234383824023795
Training accuracy : 67.51063829787235
Validation loss : 0.6712955951690673
Validation accuracy : 58.977777777777774
Epoch(41/50)
Training loss : 0.6169385732488429
Training accuracy : 69.53191489361703
Validation loss : 0.683092631234063
Validation accuracy : 57.13333333333333
Epoch(42/50)
Training loss : 0.6178911442452288
Training accuracy : 69.2127659574468
Validation loss : 0.6710708710882399
Validation accuracy : 59.22222222222222
Epoch(43/50)
Training loss : 0.6184917865915501
Training accuracy : 68.0
Validation loss : 0.6745638873842027
Validation accuracy : 56.82222222222222
Epoch(44/50)
Training loss : 0.619117021560669
Training accuracy : 69.0
Validation loss : 0.6661394794782003
Validation accuracy : 60.84444444444444
Epoch(45/50)
Training loss : 0.6170563456859994
Training accuracy : 68.2127659574468
Validation loss : 0.6732149892383151
Validation accuracy : 57.333333333333336
Epoch(46/50)
Training loss : 0.6093319819328633
Training accuracy : 70.65957446808511
Validation loss : 0.6682767113049825
Validation accuracy : 58.93333333333333
Epoch(47/50)
Training loss : 0.6125824096355033
Training accuracy : 69.80851063829788
Validation loss : 0.6730248914824591
Validation accuracy : 58.022222222222226
Epoch(48/50)
Training loss : 0.6105978780604423
Training accuracy : 69.93617021276596
Validation loss : 0.6761501736111111
Validation accuracy : 57.6
Epoch(49/50)
Training loss : 0.6114508750590872
Training accuracy : 69.70212765957447
Validation loss : 0.6694029331207275
Validation accuracy : 58.55555555555556

round(sum(accuracy_stats['train'])/len(accuracy_stats['train']),2)

63.43

import matplotlib.pyplot as plt
import seaborn as sns
train_val_acc_df = pd.DataFrame.from_dict(accuracy_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})
train_val_loss_df = pd.DataFrame.from_dict(loss_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})

fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(30,10))

sns.lineplot(data=train_val_acc_df, x = "epochs", y="value", hue="variable", ax = axes[0]).set_title('Accuracy/Epoch ')
sns.lineplot(data=train_val_loss_df, x = "epochs", y="value", hue="variable", ax = axes[1]).set_title('Loss/Epoch ')
plt.show()

y_pred_list = []
y_true_list = []
model.eval()
with torch.no_grad():
    num_correct = 0
    total = 0
    #set_trace()
    for batch, (images, labels) in enumerate(testloader,1):
        pred = model(images)
        #output = torch.exp(logps)
        pred = torch.argmax(pred, 1)
        y_pred_list.append(pred.numpy())
        y_true_list.append(labels.numpy())
        total += labels.size(0)
        num_correct += (pred == labels).sum().item()
        #print('accuracy', binary_acc(pred, labels).item())
        print(f'Batch ({batch}/{len(testloader)})')
        
        #if batch == 7:
            #break

    print(f'Accuracy of the model on {total} test images: {num_correct * 100 / total}% ')

Batch (1/49)
Batch (2/49)
Batch (3/49)
Batch (4/49)
Batch (5/49)
Batch (6/49)
Batch (7/49)
Batch (8/49)
Batch (9/49)
Batch (10/49)
Batch (11/49)
Batch (12/49)
Batch (13/49)
Batch (14/49)
Batch (15/49)
Batch (16/49)
Batch (17/49)
Batch (18/49)
Batch (19/49)
Batch (20/49)
Batch (21/49)
Batch (22/49)
Batch (23/49)
Batch (24/49)
Batch (25/49)
Batch (26/49)
Batch (27/49)
Batch (28/49)
Batch (29/49)
Batch (30/49)
Batch (31/49)
Batch (32/49)
Batch (33/49)
Batch (34/49)
Batch (35/49)
Batch (36/49)
Batch (37/49)
Batch (38/49)
Batch (39/49)
Batch (40/49)
Batch (41/49)
Batch (42/49)
Batch (43/49)
Batch (44/49)
Batch (45/49)
Batch (46/49)
Batch (47/49)
Batch (48/49)
Batch (49/49)
Accuracy of the model on 781 test images: 57.87451984635083% 

exp_name = f"Neural Network without Dropout"
expLog.loc[3,:4] = [f"{exp_name}"] + list(np.round(
               [accuracy_stats['train'][-1], 
                accuracy_stats['val'][-1],
                (num_correct * 100 / total)],3))

/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py:719: FutureWarning: Slicing a positional slice with .loc is not supported, and will raise TypeError in a future version.  Use .loc with labels or .iloc with positions instead.
  indexer = self._get_setitem_indexer(key)

y_pred_list = []
y_true_list = []
with torch.no_grad():
    for batch, (images, labels) in enumerate(testloader,1):      
        y_test_pred = model(images)
        _, y_pred_tag = torch.max(y_test_pred, dim = 1)        
        y_pred_list = [*y_pred_list,*y_pred_tag.cpu().numpy()]
        y_true_list = [*y_true_list,*labels.cpu().numpy()]

y_pred_list_1 = np.array(y_pred_list).T
#y_pred_list_1

y_true_list_1 = np.array(y_true_list).T

print(classification_report(y_true_list_1, y_pred_list_1))

              precision    recall  f1-score   support

           0       0.61      0.52      0.56       402
           1       0.56      0.64      0.60       379

    accuracy                           0.58       781
   macro avg       0.58      0.58      0.58       781
weighted avg       0.58      0.58      0.58       781

print(confusion_matrix(y_true_list_1, y_pred_list_1))

[[209 193]
 [136 243]]

idx2class = {v: k for k, v in train_it.class_to_idx.items()}
confusion_matrix_df = pd.DataFrame(confusion_matrix(y_true_list_1, y_pred_list_1)).rename(columns=idx2class, index=idx2class)
fig, ax = plt.subplots(figsize=(7,5))         
sns.heatmap(confusion_matrix_df, annot=True, ax=ax, cmap = 'Blues', fmt = 'g')
plt.show()

With Dropout

from torch.optim import Adam
class cadod(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(128*128*3, 128)
        #self.fc2 = nn.Linear(1024,512)
        #self.fc3 = nn.Linear(512,128)
        self.fc4 = nn.Linear(128,64)
        self.fc5 = nn.Linear(64,2)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(0.2)
        #self.norm1 = nn.BatchNorm1d(1024)
        self.norm2 = nn.BatchNorm1d(512)
        self.norm3 = nn.BatchNorm1d(128)
        self.norm4 = nn.BatchNorm1d(64)


    def forward(self, x):
        x = x.view(x.shape[0], -1)
        x = F.relu(self.fc1(x))
        #x = self.norm1(x)
        #x = F.relu(self.fc2(x))
        #x = self.norm2(x)
        #x = F.relu(self.fc3(x))
        x = self.norm3(x)
        x = self.dropout(x)
        x = F.relu(self.fc4(x))
        x = self.norm4(x)
        x = self.dropout(x)
        x = self.fc5(x) 
        x = F.softmax(x, dim = 1)
        return x

model = cadod()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.0002, weight_decay = 3e-3)
scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones = [500,1000,1500], gamma = 0.5)

accuracy_stats = {
    'train': [],
    "val": []
}
loss_stats = {
    'train': [],
    "val": []
}

model = model.to(device)
#optimizer = Adam(filter(lambda p: p.requires_grad, model.parameters()))

num_epochs = 50


for e in range(num_epochs):
    cum_epoch_loss = 0
    cum_acc = 0
    batch_loss = 0
    model.train()
    for batch, (images, labels) in enumerate(trainloader,1):
    #print(labels)
    #print(images.shape)
        images = images.to(device)
        labels = labels.to(device)
        optimizer.zero_grad()
        label_pred = model(images).squeeze()
        #print(logps)
        #break;
        #labels = labels.unsqueeze(1)
        loss = criterion(label_pred, labels)
        acc = binary_acc(label_pred, labels)
        loss.backward()
        optimizer.step()
        batch_loss += loss.item()
        cum_acc += acc.item()
        scheduler.step()
        #print(f'Epoch({e}/{num_epochs} : Batch number({batch}/{len(trainloader)})')
    
    with torch.no_grad():
        model.eval()
        val_epoch_loss = 0
        val_epoch_acc = 0
        for batch, (X_val_batch, y_val_batch) in enumerate(valloader,1):
            X_val_batch, y_val_batch = X_val_batch.to(device), y_val_batch.to(device)            
            y_val_pred = model(X_val_batch).squeeze()
            #y_val_pred = torch.unsqueeze(y_val_pred, 0)            
            val_loss = criterion(y_val_pred, y_val_batch)
            val_acc = binary_acc(y_val_pred, y_val_batch)            
            val_epoch_loss += val_loss.item()
            val_epoch_acc += val_acc.item()
        #print(f'Epoch({e}/{num_epochs} : Train loss : {loss.item()} : Train Accuracy : {acc.item()} : Valid loss : {val_loss.item()} : Valid Accuracy : {val_acc.item()}')
    
    loss_stats['train'].append(batch_loss/len(trainloader))
    loss_stats['val'].append(val_epoch_loss/len(valloader))
    accuracy_stats['train'].append(cum_acc/len(trainloader))
    accuracy_stats['val'].append(val_epoch_acc/len(valloader))
    
    print(f'Epoch({e}/{num_epochs})')
    print(f'Training loss : {batch_loss/len(trainloader)}')  
    print(f'Training accuracy : {cum_acc/len(trainloader)}')  
    print(f'Validation loss : {val_epoch_loss/len(valloader)}')  
    print(f'Validation accuracy : {val_epoch_acc/len(valloader)}') 
    

Epoch(0/50)
Training loss : 0.6995481886762254
Training accuracy : 53.1063829787234
Validation loss : 0.684781821568807
Validation accuracy : 56.022222222222226
Epoch(1/50)
Training loss : 0.6865724870499145
Training accuracy : 55.787234042553195
Validation loss : 0.6878685659832424
Validation accuracy : 52.77777777777778
Epoch(2/50)
Training loss : 0.6866932549375169
Training accuracy : 54.93617021276596
Validation loss : 0.6807456731796264
Validation accuracy : 56.93333333333333
Epoch(3/50)
Training loss : 0.6840045211162973
Training accuracy : 55.765957446808514
Validation loss : 0.6787495281961229
Validation accuracy : 56.888888888888886
Epoch(4/50)
Training loss : 0.6800109541162531
Training accuracy : 57.297872340425535
Validation loss : 0.6767797708511353
Validation accuracy : 55.68888888888889
Epoch(5/50)
Training loss : 0.6805504991653117
Training accuracy : 56.46808510638298
Validation loss : 0.6871784488360088
Validation accuracy : 54.6
Epoch(6/50)
Training loss : 0.6784202773520287
Training accuracy : 56.38297872340426
Validation loss : 0.6838245020972358
Validation accuracy : 55.37777777777778
Epoch(7/50)
Training loss : 0.6807573830827753
Training accuracy : 56.57446808510638
Validation loss : 0.6829860303137038
Validation accuracy : 56.77777777777778
Epoch(8/50)
Training loss : 0.6799757404530302
Training accuracy : 56.59574468085106
Validation loss : 0.6844099720319112
Validation accuracy : 56.62222222222222
Epoch(9/50)
Training loss : 0.6785362261406919
Training accuracy : 57.59574468085106
Validation loss : 0.6717781888114082
Validation accuracy : 57.22222222222222
Epoch(10/50)
Training loss : 0.6776568128707561
Training accuracy : 57.51063829787234
Validation loss : 0.6797894133461846
Validation accuracy : 56.55555555555556
Epoch(11/50)
Training loss : 0.6713103195454212
Training accuracy : 59.297872340425535
Validation loss : 0.6758932511011759
Validation accuracy : 57.666666666666664
Epoch(12/50)
Training loss : 0.6736950240236648
Training accuracy : 57.829787234042556
Validation loss : 0.6744873046875
Validation accuracy : 58.06666666666667
Epoch(13/50)
Training loss : 0.6692819899701058
Training accuracy : 59.12765957446808
Validation loss : 0.6807845301098294
Validation accuracy : 56.93333333333333
Epoch(14/50)
Training loss : 0.67269219236171
Training accuracy : 58.91489361702128
Validation loss : 0.680361184808943
Validation accuracy : 57.111111111111114
Epoch(15/50)
Training loss : 0.6684874714689052
Training accuracy : 58.808510638297875
Validation loss : 0.6754194087452359
Validation accuracy : 57.955555555555556
Epoch(16/50)
Training loss : 0.665811077077338
Training accuracy : 59.659574468085104
Validation loss : 0.6752513156996833
Validation accuracy : 57.955555555555556
Epoch(17/50)
Training loss : 0.6654714447386721
Training accuracy : 59.744680851063826
Validation loss : 0.6738522013028463
Validation accuracy : 58.31111111111111
Epoch(18/50)
Training loss : 0.668472982467489
Training accuracy : 58.46808510638298
Validation loss : 0.6745557957225375
Validation accuracy : 56.68888888888889
Epoch(19/50)
Training loss : 0.6644189205575497
Training accuracy : 60.255319148936174
Validation loss : 0.676165595319536
Validation accuracy : 58.644444444444446
Epoch(20/50)
Training loss : 0.6742742670343277
Training accuracy : 58.1063829787234
Validation loss : 0.6690668318006727
Validation accuracy : 58.888888888888886
Epoch(21/50)
Training loss : 0.6670337179873852
Training accuracy : 58.680851063829785
Validation loss : 0.6729423258039686
Validation accuracy : 58.62222222222222
Epoch(22/50)
Training loss : 0.666475809635
Training accuracy : 60.212765957446805
Validation loss : 0.675752255651686
Validation accuracy : 57.84444444444444
Epoch(23/50)
Training loss : 0.6592605887575352
Training accuracy : 61.255319148936174
Validation loss : 0.6781088444921706
Validation accuracy : 55.46666666666667
Epoch(24/50)
Training loss : 0.6586643368639844
Training accuracy : 61.851063829787236
Validation loss : 0.6745861331621806
Validation accuracy : 56.31111111111111
Epoch(25/50)
Training loss : 0.6588394591148864
Training accuracy : 61.361702127659576
Validation loss : 0.6687087112002903
Validation accuracy : 58.75555555555555
Epoch(26/50)
Training loss : 0.6622696506216171
Training accuracy : 60.97872340425532
Validation loss : 0.6713470101356507
Validation accuracy : 58.77777777777778
Epoch(27/50)
Training loss : 0.6580705021290069
Training accuracy : 61.0
Validation loss : 0.6754859142833286
Validation accuracy : 57.422222222222224
Epoch(28/50)
Training loss : 0.6587620428267945
Training accuracy : 61.12765957446808
Validation loss : 0.6703884085019429
Validation accuracy : 59.111111111111114
Epoch(29/50)
Training loss : 0.655924732380725
Training accuracy : 61.1063829787234
Validation loss : 0.6786305851406521
Validation accuracy : 57.24444444444445
Epoch(30/50)
Training loss : 0.6581049422000317
Training accuracy : 60.191489361702125
Validation loss : 0.6761952784326342
Validation accuracy : 56.355555555555554
Epoch(31/50)
Training loss : 0.6604712301112236
Training accuracy : 59.787234042553195
Validation loss : 0.6641817079650031
Validation accuracy : 60.06666666666667
Epoch(32/50)
Training loss : 0.6575885098031227
Training accuracy : 60.808510638297875
Validation loss : 0.6701030797428555
Validation accuracy : 57.51111111111111
Epoch(33/50)
Training loss : 0.6555307862606454
Training accuracy : 62.02127659574468
Validation loss : 0.6709122459093729
Validation accuracy : 57.666666666666664
Epoch(34/50)
Training loss : 0.6553964576822646
Training accuracy : 62.276595744680854
Validation loss : 0.6718618869781494
Validation accuracy : 57.8
Epoch(35/50)
Training loss : 0.6566752842132081
Training accuracy : 61.59574468085106
Validation loss : 0.666807124349806
Validation accuracy : 57.84444444444444
Epoch(36/50)
Training loss : 0.6520871837088402
Training accuracy : 62.297872340425535
Validation loss : 0.6669018096394009
Validation accuracy : 59.15555555555556
Epoch(37/50)
Training loss : 0.647441896986454
Training accuracy : 62.5531914893617
Validation loss : 0.6758612487051222
Validation accuracy : 57.68888888888889
Epoch(38/50)
Training loss : 0.6503621149570384
Training accuracy : 62.46808510638298
Validation loss : 0.6744203779432508
Validation accuracy : 56.86666666666667
Epoch(39/50)
Training loss : 0.6498629628343785
Training accuracy : 61.8936170212766
Validation loss : 0.6722906827926636
Validation accuracy : 59.71111111111111
Epoch(40/50)
Training loss : 0.6450835948294782
Training accuracy : 63.829787234042556
Validation loss : 0.6659585820304023
Validation accuracy : 59.71111111111111
Epoch(41/50)
Training loss : 0.6479960641962417
Training accuracy : 62.87234042553192
Validation loss : 0.672738167974684
Validation accuracy : 60.111111111111114
Epoch(42/50)
Training loss : 0.6492712383574628
Training accuracy : 62.12765957446808
Validation loss : 0.6649191035164728
Validation accuracy : 58.44444444444444
Epoch(43/50)
Training loss : 0.6480792862303714
Training accuracy : 63.659574468085104
Validation loss : 0.673744277159373
Validation accuracy : 58.46666666666667
Epoch(44/50)
Training loss : 0.6487881909025476
Training accuracy : 63.638297872340424
Validation loss : 0.6695589198006524
Validation accuracy : 57.333333333333336
Epoch(45/50)
Training loss : 0.6491463780403137
Training accuracy : 63.170212765957444
Validation loss : 0.6695530984136794
Validation accuracy : 58.888888888888886
Epoch(46/50)
Training loss : 0.6435056087818551
Training accuracy : 62.91489361702128
Validation loss : 0.6726370374361674
Validation accuracy : 58.13333333333333
Epoch(47/50)
Training loss : 0.6467280387878418
Training accuracy : 63.744680851063826
Validation loss : 0.6686036275492774
Validation accuracy : 57.51111111111111
Epoch(48/50)
Training loss : 0.6431198462526849
Training accuracy : 63.702127659574465
Validation loss : 0.663936189810435
Validation accuracy : 59.06666666666667
Epoch(49/50)
Training loss : 0.6434404025686548
Training accuracy : 63.829787234042556
Validation loss : 0.6686546153492398
Validation accuracy : 57.644444444444446

train_val_acc_df = pd.DataFrame.from_dict(accuracy_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})
train_val_loss_df = pd.DataFrame.from_dict(loss_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})

fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(30,10))

sns.lineplot(data=train_val_acc_df, x = "epochs", y="value", hue="variable", ax = axes[0]).set_title('Accuracy/Epoch')
sns.lineplot(data=train_val_loss_df, x = "epochs", y="value", hue="variable", ax = axes[1]).set_title('Loss/Epoch')
plt.show()

y_pred_list = []
y_true_list = []
model.eval()
with torch.no_grad():
    num_correct = 0
    total = 0
    #set_trace()
    for batch, (images, labels) in enumerate(testloader,1):
        pred = model(images)
        #output = torch.exp(logps)
        pred = torch.argmax(pred, 1)
        y_pred_list.append(pred.numpy())
        y_true_list.append(labels.numpy())
        total += labels.size(0)
        num_correct += (pred == labels).sum().item()
        #print('accuracy', binary_acc(pred, labels).item())
        print(f'Batch ({batch}/{len(testloader)})')
        
        #if batch == 7:
            #break

    print(f'Accuracy of the model on {total} test images: {num_correct * 100 / total}% ')

Batch (1/49)
Batch (2/49)
Batch (3/49)
Batch (4/49)
Batch (5/49)
Batch (6/49)
Batch (7/49)
Batch (8/49)
Batch (9/49)
Batch (10/49)
Batch (11/49)
Batch (12/49)
Batch (13/49)
Batch (14/49)
Batch (15/49)
Batch (16/49)
Batch (17/49)
Batch (18/49)
Batch (19/49)
Batch (20/49)
Batch (21/49)
Batch (22/49)
Batch (23/49)
Batch (24/49)
Batch (25/49)
Batch (26/49)
Batch (27/49)
Batch (28/49)
Batch (29/49)
Batch (30/49)
Batch (31/49)
Batch (32/49)
Batch (33/49)
Batch (34/49)
Batch (35/49)
Batch (36/49)
Batch (37/49)
Batch (38/49)
Batch (39/49)
Batch (40/49)
Batch (41/49)
Batch (42/49)
Batch (43/49)
Batch (44/49)
Batch (45/49)
Batch (46/49)
Batch (47/49)
Batch (48/49)
Batch (49/49)
Accuracy of the model on 781 test images: 58.25864276568502% 

y_pred_list = []
y_true_list = []
with torch.no_grad():
    for batch, (images, labels) in enumerate(testloader,1):      
        y_test_pred = model(images)
        _, y_pred_tag = torch.max(y_test_pred, dim = 1)        
        y_pred_list = [*y_pred_list,*y_pred_tag.cpu().numpy()]
        y_true_list = [*y_true_list,*labels.cpu().numpy()]

exp_name = f"Neural Network with Dropout (p = 0.2)"
expLog.loc[4,:4] = [f"{exp_name}"] + list(np.round(
               [accuracy_stats['train'][-1], 
                accuracy_stats['val'][-1],
                (num_correct * 100 / total)],3))
expLog

/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py:719: FutureWarning: Slicing a positional slice with .loc is not supported, and will raise TypeError in a future version.  Use .loc with labels or .iloc with positions instead.
  indexer = self._get_setitem_indexer(key)

	exp_name	Train Acc	Valid Acc	Test Acc
0	Baseline Neural Network without Augmentation, ...	97.9	56.022	56.978
1	Neural Network without Augmentation and with R...	96.325	56.467	57.106
2	Neural Network without Regularization and Dropout	70.702	57.511	59.923
3	Neural Network without Dropout	69.702	58.556	57.875
4	Neural Network with Dropout (p = 0.2)	63.83	57.644	58.259

y_pred_list_1 = np.array(y_pred_list).T
y_true_list_1 = np.array(y_true_list).T
#y_pred_list_1

print(classification_report(y_true_list_1, y_pred_list_1))

              precision    recall  f1-score   support

           0       0.59      0.62      0.61       402
           1       0.57      0.54      0.56       379

    accuracy                           0.58       781
   macro avg       0.58      0.58      0.58       781
weighted avg       0.58      0.58      0.58       781

print(confusion_matrix(y_true_list_1, y_pred_list_1))

[[251 151]
 [175 204]]

idx2class = {v: k for k, v in train_it.class_to_idx.items()}
confusion_matrix_df = pd.DataFrame(confusion_matrix(y_true_list_1, y_pred_list_1)).rename(columns=idx2class, index=idx2class)
fig, ax = plt.subplots(figsize=(7,5))         
sns.heatmap(confusion_matrix_df, annot=True, ax=ax, cmap = 'Blues', fmt = 'g')
plt.show()

Drop out value of 0.1

from torch.optim import Adam
class cadod(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(128*128*3, 128)
        #self.fc2 = nn.Linear(1024,512)
        #self.fc3 = nn.Linear(512,128)
        self.fc4 = nn.Linear(128,64)
        self.fc5 = nn.Linear(64,2)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(0.1)
        #self.norm1 = nn.BatchNorm1d(1024)
        self.norm2 = nn.BatchNorm1d(512)
        self.norm3 = nn.BatchNorm1d(128)
        self.norm4 = nn.BatchNorm1d(64)


    def forward(self, x):
        x = x.view(x.shape[0], -1)
        x = F.relu(self.fc1(x))
        #x = self.norm1(x)
        #x = F.relu(self.fc2(x))
        #x = self.norm2(x)
        #x = F.relu(self.fc3(x))
        x = self.norm3(x)
        x = self.dropout(x)
        x = F.relu(self.fc4(x))
        x = self.norm4(x)
        x =self.dropout(x)
        x = self.fc5(x) 
        x = F.softmax(x, dim = 1)
        return x

model = cadod()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.0002, weight_decay = 3e-3)
scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones = [500,1000,1500], gamma = 0.5)

accuracy_stats = {
    'train': [],
    "val": []
}
loss_stats = {
    'train': [],
    "val": []
}

model = model.to(device)
#optimizer = Adam(filter(lambda p: p.requires_grad, model.parameters()))

num_epochs = 50


for e in range(num_epochs):
    cum_epoch_loss = 0
    cum_acc = 0
    batch_loss = 0
    model.train()
    for batch, (images, labels) in enumerate(trainloader,1):
    #print(labels)
    #print(images.shape)
        images = images.to(device)
        labels = labels.to(device)
        optimizer.zero_grad()
        label_pred = model(images).squeeze()
        #print(logps)
        #break;
        #labels = labels.unsqueeze(1)
        loss = criterion(label_pred, labels)
        acc = binary_acc(label_pred, labels)
        loss.backward()
        optimizer.step()
        batch_loss += loss.item()
        cum_acc += acc.item()
        scheduler.step()
        #print(f'Epoch({e}/{num_epochs} : Batch number({batch}/{len(trainloader)})')
    
    with torch.no_grad():
        model.eval()
        val_epoch_loss = 0
        val_epoch_acc = 0
        for batch, (X_val_batch, y_val_batch) in enumerate(valloader,1):
            X_val_batch, y_val_batch = X_val_batch.to(device), y_val_batch.to(device)            
            y_val_pred = model(X_val_batch).squeeze()
            #y_val_pred = torch.unsqueeze(y_val_pred, 0)            
            val_loss = criterion(y_val_pred, y_val_batch)
            val_acc = binary_acc(y_val_pred, y_val_batch)            
            val_epoch_loss += val_loss.item()
            val_epoch_acc += val_acc.item()
        #print(f'Epoch({e}/{num_epochs} : Train loss : {loss.item()} : Train Accuracy : {acc.item()} : Valid loss : {val_loss.item()} : Valid Accuracy : {val_acc.item()}')
    
    loss_stats['train'].append(batch_loss/len(trainloader))
    loss_stats['val'].append(val_epoch_loss/len(valloader))
    accuracy_stats['train'].append(cum_acc/len(trainloader))
    accuracy_stats['val'].append(val_epoch_acc/len(valloader))
    
    print(f'Epoch({e}/{num_epochs})')
    print(f'Training loss : {batch_loss/len(trainloader)}')  
    print(f'Training accuracy : {cum_acc/len(trainloader)}')  
    print(f'Validation loss : {val_epoch_loss/len(valloader)}')  
    print(f'Validation accuracy : {val_epoch_acc/len(valloader)}') 
    

Epoch(0/50)
Training loss : 0.7019766556455734
Training accuracy : 52.276595744680854
Validation loss : 0.6893682426876492
Validation accuracy : 54.55555555555556
Epoch(1/50)
Training loss : 0.6873131483159167
Training accuracy : 54.46808510638298
Validation loss : 0.6847085224257575
Validation accuracy : 55.51111111111111
Epoch(2/50)
Training loss : 0.6799716543644032
Training accuracy : 56.91489361702128
Validation loss : 0.6892028093338013
Validation accuracy : 53.55555555555556
Epoch(3/50)
Training loss : 0.6879159021884838
Training accuracy : 55.319148936170215
Validation loss : 0.6809960431522794
Validation accuracy : 55.955555555555556
Epoch(4/50)
Training loss : 0.6777490770563166
Training accuracy : 57.808510638297875
Validation loss : 0.6807383153173658
Validation accuracy : 56.48888888888889
Epoch(5/50)
Training loss : 0.67876409850222
Training accuracy : 57.42553191489362
Validation loss : 0.6690863993432786
Validation accuracy : 60.55555555555556
Epoch(6/50)
Training loss : 0.675690491148766
Training accuracy : 58.8936170212766
Validation loss : 0.6813991904258728
Validation accuracy : 54.77777777777778
Epoch(7/50)
Training loss : 0.6782930239718011
Training accuracy : 56.680851063829785
Validation loss : 0.6857803331481086
Validation accuracy : 56.955555555555556
Epoch(8/50)
Training loss : 0.6751730771774941
Training accuracy : 57.08510638297872
Validation loss : 0.6773703349961175
Validation accuracy : 57.644444444444446
Epoch(9/50)
Training loss : 0.6719924553911737
Training accuracy : 58.5531914893617
Validation loss : 0.6786959502432082
Validation accuracy : 56.977777777777774
Epoch(10/50)
Training loss : 0.6714132136486947
Training accuracy : 58.148936170212764
Validation loss : 0.6773722781075372
Validation accuracy : 56.31111111111111
Epoch(11/50)
Training loss : 0.6705846583589594
Training accuracy : 58.87234042553192
Validation loss : 0.6803095287746853
Validation accuracy : 57.022222222222226
Epoch(12/50)
Training loss : 0.6660362291843334
Training accuracy : 59.95744680851064
Validation loss : 0.6770708825853136
Validation accuracy : 54.733333333333334
Epoch(13/50)
Training loss : 0.6642993815401768
Training accuracy : 58.97872340425532
Validation loss : 0.6771332846747504
Validation accuracy : 58.84444444444444
Epoch(14/50)
Training loss : 0.6695615205358951
Training accuracy : 58.93617021276596
Validation loss : 0.6709088961283366
Validation accuracy : 59.355555555555554
Epoch(15/50)
Training loss : 0.6672486421909738
Training accuracy : 58.851063829787236
Validation loss : 0.6685203168127272
Validation accuracy : 59.44444444444444
Epoch(16/50)
Training loss : 0.6648394414719115
Training accuracy : 60.148936170212764
Validation loss : 0.677607696586185
Validation accuracy : 57.977777777777774
Epoch(17/50)
Training loss : 0.663974164648259
Training accuracy : 61.319148936170215
Validation loss : 0.6717222293217977
Validation accuracy : 59.37777777777778
Epoch(18/50)
Training loss : 0.6624360604489103
Training accuracy : 60.255319148936174
Validation loss : 0.6714106294843886
Validation accuracy : 57.577777777777776
Epoch(19/50)
Training loss : 0.6650161058344739
Training accuracy : 59.8936170212766
Validation loss : 0.6730354825655619
Validation accuracy : 57.91111111111111
Epoch(20/50)
Training loss : 0.6636931820118681
Training accuracy : 59.8936170212766
Validation loss : 0.671837219927046
Validation accuracy : 57.93333333333333
Epoch(21/50)
Training loss : 0.6624099181053487
Training accuracy : 60.61702127659574
Validation loss : 0.6710527936617533
Validation accuracy : 59.44444444444444
Epoch(22/50)
Training loss : 0.6604233688496529
Training accuracy : 60.87234042553192
Validation loss : 0.6725348419613308
Validation accuracy : 58.266666666666666
Epoch(23/50)
Training loss : 0.6528097200900951
Training accuracy : 62.808510638297875
Validation loss : 0.6710558427704705
Validation accuracy : 58.2
Epoch(24/50)
Training loss : 0.6513950190645583
Training accuracy : 62.97872340425532
Validation loss : 0.6647563033633762
Validation accuracy : 60.888888888888886
Epoch(25/50)
Training loss : 0.6532557999834101
Training accuracy : 62.5531914893617
Validation loss : 0.67105041080051
Validation accuracy : 56.333333333333336
Epoch(26/50)
Training loss : 0.6599491999504414
Training accuracy : 61.1063829787234
Validation loss : 0.678892523712582
Validation accuracy : 59.111111111111114
Epoch(27/50)
Training loss : 0.6538680487490715
Training accuracy : 62.148936170212764
Validation loss : 0.6814597527186076
Validation accuracy : 54.93333333333333
Epoch(28/50)
Training loss : 0.6513881696031448
Training accuracy : 62.8936170212766
Validation loss : 0.6730901334020827
Validation accuracy : 58.111111111111114
Epoch(29/50)
Training loss : 0.6517227494970281
Training accuracy : 61.723404255319146
Validation loss : 0.6661439286337958
Validation accuracy : 59.644444444444446
Epoch(30/50)
Training loss : 0.654126654279993
Training accuracy : 62.234042553191486
Validation loss : 0.6738883031739129
Validation accuracy : 57.577777777777776
Epoch(31/50)
Training loss : 0.6565782871652157
Training accuracy : 61.744680851063826
Validation loss : 0.6654651721318563
Validation accuracy : 59.044444444444444
Epoch(32/50)
Training loss : 0.6442193160665796
Training accuracy : 63.53191489361702
Validation loss : 0.6666519853803846
Validation accuracy : 59.888888888888886
Epoch(33/50)
Training loss : 0.6408722666983909
Training accuracy : 65.17021276595744
Validation loss : 0.6694507996241251
Validation accuracy : 58.91111111111111
Epoch(34/50)
Training loss : 0.6424177479236683
Training accuracy : 64.7872340425532
Validation loss : 0.666528520319197
Validation accuracy : 59.4
Epoch(35/50)
Training loss : 0.6379395697979217
Training accuracy : 65.51063829787235
Validation loss : 0.6681166542900934
Validation accuracy : 59.6
Epoch(36/50)
Training loss : 0.6411497326607399
Training accuracy : 63.787234042553195
Validation loss : 0.6697189821137323
Validation accuracy : 57.955555555555556
Epoch(37/50)
Training loss : 0.637284461488115
Training accuracy : 65.40425531914893
Validation loss : 0.6618572910626729
Validation accuracy : 59.111111111111114
Epoch(38/50)
Training loss : 0.6348496104808564
Training accuracy : 64.91489361702128
Validation loss : 0.6692223621739282
Validation accuracy : 58.62222222222222
Epoch(39/50)
Training loss : 0.6387967327807812
Training accuracy : 63.87234042553192
Validation loss : 0.671270740032196
Validation accuracy : 59.48888888888889
Epoch(40/50)
Training loss : 0.6365515183895192
Training accuracy : 65.46808510638297
Validation loss : 0.6701709826787313
Validation accuracy : 57.15555555555556
Epoch(41/50)
Training loss : 0.6320443762109634
Training accuracy : 66.04255319148936
Validation loss : 0.6744220402505663
Validation accuracy : 58.111111111111114
Epoch(42/50)
Training loss : 0.6333851547951393
Training accuracy : 66.0
Validation loss : 0.6759532544347975
Validation accuracy : 57.77777777777778
Epoch(43/50)
Training loss : 0.6381754380591372
Training accuracy : 64.70212765957447
Validation loss : 0.670289142926534
Validation accuracy : 57.68888888888889
Epoch(44/50)
Training loss : 0.6311595858411586
Training accuracy : 66.19148936170212
Validation loss : 0.6709285762574938
Validation accuracy : 58.333333333333336
Epoch(45/50)
Training loss : 0.6370452363440331
Training accuracy : 64.55319148936171
Validation loss : 0.6681258413526747
Validation accuracy : 58.17777777777778
Epoch(46/50)
Training loss : 0.6326577168829898
Training accuracy : 65.08510638297872
Validation loss : 0.6713515890969171
Validation accuracy : 57.53333333333333
Epoch(47/50)
Training loss : 0.6266423552594287
Training accuracy : 66.51063829787235
Validation loss : 0.6669643057717217
Validation accuracy : 58.31111111111111
Epoch(48/50)
Training loss : 0.6314820571148649
Training accuracy : 65.72340425531915
Validation loss : 0.6663783431053162
Validation accuracy : 59.888888888888886
Epoch(49/50)
Training loss : 0.6286288360331921
Training accuracy : 66.82978723404256
Validation loss : 0.6733958522478739
Validation accuracy : 59.08888888888889

train_val_acc_df = pd.DataFrame.from_dict(accuracy_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})
train_val_loss_df = pd.DataFrame.from_dict(loss_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})

fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(30,10))

sns.lineplot(data=train_val_acc_df, x = "epochs", y="value", hue="variable", ax = axes[0]).set_title('Accuracy/Epoch')
sns.lineplot(data=train_val_loss_df, x = "epochs", y="value", hue="variable", ax = axes[1]).set_title('Loss/Epoch')
plt.show()

y_pred_list = []
y_true_list = []
model.eval()
with torch.no_grad():
    num_correct = 0
    total = 0
    #set_trace()
    for batch, (images, labels) in enumerate(testloader,1):
        pred = model(images)
        #output = torch.exp(logps)
        pred = torch.argmax(pred, 1)
        y_pred_list.append(pred.numpy())
        y_true_list.append(labels.numpy())
        total += labels.size(0)
        num_correct += (pred == labels).sum().item()
        #print('accuracy', binary_acc(pred, labels).item())
        print(f'Batch ({batch}/{len(testloader)})')
        
        #if batch == 7:
            #break

    print(f'Accuracy of the model on {total} test images: {num_correct * 100 / total}% ')

Batch (1/49)
Batch (2/49)
Batch (3/49)
Batch (4/49)
Batch (5/49)
Batch (6/49)
Batch (7/49)
Batch (8/49)
Batch (9/49)
Batch (10/49)
Batch (11/49)
Batch (12/49)
Batch (13/49)
Batch (14/49)
Batch (15/49)
Batch (16/49)
Batch (17/49)
Batch (18/49)
Batch (19/49)
Batch (20/49)
Batch (21/49)
Batch (22/49)
Batch (23/49)
Batch (24/49)
Batch (25/49)
Batch (26/49)
Batch (27/49)
Batch (28/49)
Batch (29/49)
Batch (30/49)
Batch (31/49)
Batch (32/49)
Batch (33/49)
Batch (34/49)
Batch (35/49)
Batch (36/49)
Batch (37/49)
Batch (38/49)
Batch (39/49)
Batch (40/49)
Batch (41/49)
Batch (42/49)
Batch (43/49)
Batch (44/49)
Batch (45/49)
Batch (46/49)
Batch (47/49)
Batch (48/49)
Batch (49/49)
Accuracy of the model on 781 test images: 59.28297055057619% 

y_pred_list = []
y_true_list = []
with torch.no_grad():
    for batch, (images, labels) in enumerate(testloader,1):      
        y_test_pred = model(images)
        _, y_pred_tag = torch.max(y_test_pred, dim = 1)        
        y_pred_list = [*y_pred_list,*y_pred_tag.cpu().numpy()]
        y_true_list = [*y_true_list,*labels.cpu().numpy()]

y_pred_list_1 = np.array(y_pred_list).T
y_true_list_1 = np.array(y_true_list).T
#y_pred_list_1

exp_name = f"Neural Network with Dropout (p = 0.1)"
expLog.loc[5,:4] = [f"{exp_name}"] + list(np.round(
               [accuracy_stats['train'][-1], 
                accuracy_stats['val'][-1],
                (num_correct * 100 / total)],3))
expLog

/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py:719: FutureWarning: Slicing a positional slice with .loc is not supported, and will raise TypeError in a future version.  Use .loc with labels or .iloc with positions instead.
  indexer = self._get_setitem_indexer(key)

	exp_name	Train Acc	Valid Acc	Test Acc
0	Baseline Neural Network without Augmentation, ...	97.9	56.022	56.978
1	Neural Network without Augmentation and with R...	96.325	56.467	57.106
2	Neural Network without Regularization and Dropout	70.702	57.511	59.923
3	Neural Network without Dropout	69.702	58.556	57.875
4	Neural Network with Dropout (p = 0.2)	63.83	57.644	58.259
5	Neural Network with Dropout (p = 0.1)	66.83	59.089	59.283

y_pred_list_1 = np.array(y_pred_list).T
y_true_list_1 = np.array(y_true_list).T
#y_pred_list_1

print(classification_report(y_true_list_1, y_pred_list_1))

              precision    recall  f1-score   support

           0       0.61      0.57      0.59       402
           1       0.57      0.62      0.60       379

    accuracy                           0.59       781
   macro avg       0.59      0.59      0.59       781
weighted avg       0.59      0.59      0.59       781

print(confusion_matrix(y_true_list_1, y_pred_list_1))

[[229 173]
 [145 234]]

idx2class = {v: k for k, v in train_it.class_to_idx.items()}
confusion_matrix_df = pd.DataFrame(confusion_matrix(y_true_list_1, y_pred_list_1)).rename(columns=idx2class, index=idx2class)
fig, ax = plt.subplots(figsize=(7,5))         
sns.heatmap(confusion_matrix_df, annot=True, ax=ax, cmap = 'Blues', fmt = 'g')
plt.show()

Results and Discussion

• Without augmentation, we see that though we got a training accuracy of more than 90%, the validation accuracy of only 56%, which clearly indicates that the model is overfitting the data

• Adding regularization did drop the test accuracy but not to a larger extent

• We have overcome this by data augmentation, adding drop out layers and regularization

• Our best model with a dropout layer of 0.1 yielded us a training accuracy of 67% and validation and test accuracy of ~60%

• We also see that we also got a f1 score of ~60% for this model, with close to no difference in precision and recall values

MLP Regressor for Bounding Box prediction

Sequential neural network regressor

• Loss function used – MSE

• We used a one input layer, 2 hidden layers, and an output layer in our neural network

• Tanh was used the activation function

• We also added multiple dropout layers to overcome overfitting

• We have used Adam as our optimizer with a learning rate of

• We also used a weight decay value of 1e-3 which will provide us with L2 regularization

A similar process was followed for OOP as well

import torch
import torchvision
import torch.utils.data
import torchvision.transforms as transforms
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_absolute_error, mean_squared_error

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
device

device(type='cpu')

from google.colab import drive drive.mount('/content/drive')

!ls "/content/drive/My Drive/Colab Notebooks"

import numpy as np
X = np.load('data/img.npy', allow_pickle=True)
y_bbox = np.load('data/y_bbox.npy', allow_pickle=True)

print(X.shape, y_bbox.shape)

(12966, 49152) (12966, 4)

X_train, X_test, y_train, y_test = train_test_split(X, y_bbox, test_size=0.15, random_state=42)
X_train, X_validation, y_train, y_validation = train_test_split(X_train, y_train, test_size=0.15, random_state=42)

print(X_train.shape, y_train.shape)

(9367, 49152) (9367, 4)

!pip install torchsummary 

Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: torchsummary in /N/home/u080/schakkad/Carbonate/.local/lib/python3.7/site-packages (1.5.1)

pip install --upgrade pip

Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pip in /N/home/u080/schakkad/Carbonate/.local/lib/python3.7/site-packages (21.3.1)
Note: you may need to restart the kernel to use updated packages.

from torchsummary import summary  #install it if necessary using !pip install torchsummary 
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler


## Scaling
scaler = StandardScaler()
X_train =      scaler.fit_transform(X_train).astype(float)
X_validation = scaler.transform(X_validation).astype(float) #Transform valid set with the same constants
X_test =       scaler.transform(X_test).astype(float)       #Transform test  set with the same constants

# convert numpy arrays to tensors
X_train_tensor = torch.from_numpy(X_train).float()
X_validation_tensor = torch.from_numpy(X_validation).float()
X_test_tensor = torch.from_numpy(X_test).float()
y_train_tensor = torch.from_numpy(y_train).float()
y_test_tensor = torch.from_numpy(y_test).float()
y_validation_tensor = torch.from_numpy(y_validation).float()

# create TensorDataset in PyTorch
train_ds = torch.utils.data.TensorDataset(X_train_tensor, y_train_tensor)
validation_ds = torch.utils.data.TensorDataset(X_validation_tensor, y_validation_tensor)
test_ds = torch.utils.data.TensorDataset(X_test_tensor, y_test_tensor)
# create dataloader
batch_size = 96
train_loader = torch.utils.data.DataLoader(train_ds, batch_size=batch_size, shuffle=True, num_workers=0)
valid_loader = torch.utils.data.DataLoader(validation_ds, batch_size=X_test.shape[0], shuffle=False, num_workers=0)
test_loader = torch.utils.data.DataLoader(test_ds, batch_size=X_test.shape[0], shuffle=False, num_workers=0)

X_train.shape

(9367, 49152)

y_train.shape

(9367, 4)

D_in

49152

model = torch.nn.Sequential()

# Use the nn package to define our model and loss function.
# use the sequential API to create a neural network regressor

D_in = X_test.shape[1]
print(D_in)
D_hidden_1 =128
D_hidden_2 = 64
D_out = 4

model = torch.nn.Sequential(
    
    torch.nn.Linear(D_in, D_hidden_1),   
    nn.Tanh(),
    nn.Linear(D_hidden_1, D_hidden_2),
    nn.Tanh(),
    nn.Linear(in_features=D_hidden_2, out_features=D_out), 
    nn.Dropout(0.1)
    
)

     
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
# Assuming that we are on a CUDA machine, this should print a CUDA device:
print(f"We are working on a {device} device")


model.to(device) # put on GPU before setting up the optimizer

49152
We are working on a cpu device

Sequential(
  (0): Linear(in_features=49152, out_features=128, bias=True)
  (1): Tanh()
  (2): Linear(in_features=128, out_features=64, bias=True)
  (3): Tanh()
  (4): Linear(in_features=64, out_features=4, bias=True)
  (5): Dropout(p=0.1, inplace=False)
)

model.to(device) 

Sequential(
  (0): Linear(in_features=49152, out_features=128, bias=True)
  (1): Tanh()
  (2): Linear(in_features=128, out_features=64, bias=True)
  (3): Tanh()
  (4): Linear(in_features=64, out_features=4, bias=True)
  (5): Dropout(p=0.1, inplace=False)
)

loss_stats = {
    'train': [],
    "val": []
}

loss_fn = torch.nn.MSELoss(size_average=True)

optimizer = optim.Adam(model.parameters(), lr=0.0001, weight_decay = 0.005)

epochs = range(25)
count = 0 
running_loss = 0.0
for epoch in epochs:
    running_loss = 0.0
    for batch, data in enumerate(train_loader):
        # inputs, target = data[0].to(device), data[1].to(device)
        inputs, target = data[0], data[1]

        # Clear gradient buffers because we don't want any gradient from previous epoch to carry forward, dont want to cummulate gradients
        optimizer.zero_grad()

        # do forward pass
        output = model(inputs.float())
        #print(output.shape, target.shape)
        # compute loss and gradients
        loss = loss_fn(output, torch.unsqueeze(target.float(), dim=1))
        
        # get gradients w.r.t to parameters
        loss.backward()

        # perform gradient update
        optimizer.step()

        # print statistics
        running_loss += loss.item()*inputs.size(0)
        count += inputs.size(0)
    print(f"Epoch {epoch+1}, Test MSE loss: {np.round(running_loss/count, 3)}")
    loss_stats['train'].append(running_loss/count)

#print('Finished Training')
#Model evaluation on validation set
    count = 0 
    running_loss = 0.0
    test_size = 0
    model.eval()
    for batch, data in enumerate(valid_loader):
       
        inputs, target = data[0], data[1]
        # do forward pass
        output = model(inputs.float())
        # compute loss and gradients
        loss = loss_fn(output, torch.unsqueeze(target.float(), dim=1))
        running_loss += loss.item()*inputs.size(0)
        count += inputs.size(0) 
        test_size += batch_size

    print(f"Validation  MSE loss: {np.round(running_loss/count, 3)}")
    
    loss_stats['val'].append(running_loss/count)

print("Finished Training!")

count = 0 
running_loss = 0.0
test_size = 0
model.eval()
for batch, data in enumerate(test_loader):
    
    inputs, target = data[0], data[1]
    # do forward pass
    output = model(inputs.float())
    # compute loss and gradients
    loss = loss_fn(output, torch.unsqueeze(target.float(), dim=1))
    # print statistics
    running_loss += loss.item()*inputs.size(0)
    count += inputs.size(0) 
    test_size += batch_size
print(f" Test  MSE loss: {np.round(running_loss/count, 3)}")

# predict test
# model.to(device)

/usr/local/lib/python3.7/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.
  warnings.warn(warning.format(ret))
/usr/local/lib/python3.7/site-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([96, 1, 4])) that is different to the input size (torch.Size([96, 4])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([55, 1, 4])) that is different to the input size (torch.Size([55, 4])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)

Epoch 1, Test MSE loss: 0.524

/usr/local/lib/python3.7/site-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([1654, 1, 4])) that is different to the input size (torch.Size([1654, 4])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)

Validation  MSE loss: 0.456
Epoch 2, Test MSE loss: 0.359
Validation  MSE loss: 0.406
Epoch 3, Test MSE loss: 0.31
Validation  MSE loss: 0.352
Epoch 4, Test MSE loss: 0.261
Validation  MSE loss: 0.299
Epoch 5, Test MSE loss: 0.21
Validation  MSE loss: 0.243
Epoch 6, Test MSE loss: 0.158
Validation  MSE loss: 0.189
Epoch 7, Test MSE loss: 0.119
Validation  MSE loss: 0.153
Epoch 8, Test MSE loss: 0.089
Validation  MSE loss: 0.12
Epoch 9, Test MSE loss: 0.065
Validation  MSE loss: 0.095
Epoch 10, Test MSE loss: 0.049
Validation  MSE loss: 0.078
Epoch 11, Test MSE loss: 0.038
Validation  MSE loss: 0.062
Epoch 12, Test MSE loss: 0.03
Validation  MSE loss: 0.051
Epoch 13, Test MSE loss: 0.024
Validation  MSE loss: 0.042
Epoch 14, Test MSE loss: 0.02
Validation  MSE loss: 0.036
Epoch 15, Test MSE loss: 0.017
Validation  MSE loss: 0.03
Epoch 16, Test MSE loss: 0.015
Validation  MSE loss: 0.026
Epoch 17, Test MSE loss: 0.013
Validation  MSE loss: 0.024
Epoch 18, Test MSE loss: 0.013
Validation  MSE loss: 0.02
Epoch 19, Test MSE loss: 0.012
Validation  MSE loss: 0.018
Epoch 20, Test MSE loss: 0.011
Validation  MSE loss: 0.017
Epoch 21, Test MSE loss: 0.01
Validation  MSE loss: 0.015
Epoch 22, Test MSE loss: 0.01
Validation  MSE loss: 0.014
Epoch 23, Test MSE loss: 0.01
Validation  MSE loss: 0.014
Epoch 24, Test MSE loss: 0.01
Validation  MSE loss: 0.012
Epoch 25, Test MSE loss: 0.01
Validation  MSE loss: 0.012
Finished Training!
 Test  MSE loss: 0.012

/usr/local/lib/python3.7/site-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([1945, 1, 4])) that is different to the input size (torch.Size([1945, 4])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)

#Plotting train and avlidation MSE
import seaborn as sns
import matplotlib.pyplot as plt
train_val_loss_df = pd.DataFrame.from_dict(loss_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})

#fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(30,10))

sns.lineplot(data=train_val_loss_df, x = "epochs", y="value", hue="variable").set_title('Loss/Epoch')
plt.xlim([0,25])
plt.show()

expLog_R = pd.DataFrame(columns=["exp_name", 
                               "Train MSE", 
                               "Valid MSE",
                               "Test  MSE",
                              ])

#Logging the experiment
exp_name = f"Sequential Regression NN without dropout for weight_decay = 0.05"
expLog_R.loc[8,:4] = [f"{exp_name}"] + list(np.round(
               [loss_stats['train'][-1], 
                loss_stats['val'][-1],
                running_loss/count],3))
expLog_R

/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py:719: FutureWarning: Slicing a positional slice with .loc is not supported, and will raise TypeError in a future version.  Use .loc with labels or .iloc with positions instead.
  indexer = self._get_setitem_indexer(key)

	exp_name	Train MSE	Valid MSE	Test MSE
2	Sequential Regression NN Dropout (p = 0.5)	0.009	0.011	0.011
0	Sequential Regression NN Dropout (p = 0.1)	0.009	0.011	0.011
1	Sequential Regression NN Dropout (p = 0.2)	0.01	0.012	0.012
3	Sequential Regression NN Dropout (p = 0.3)	0.01	0.013	0.013
4	Sequential Regression NN Dropout (p = 0.4)	0.01	0.012	0.012
5	Sequential Regression NN without dropout	0.01	0.012	0.012
6	Sequential Regression NN without dropout for w...	0.009	0.011	0.01
7	Sequential Regression NN without dropout for w...	0.01	0.022	0.022
8	Sequential Regression NN without dropout for w...	0.01	0.012	0.012

OOP API for Regression

D_in = 128*128*3
print(D_in)
D_hidden = 128
D_out = 4
# Use the nn package to define our model and loss function.
# use the sequential API makes things simple
class cadod_r(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(D_in, 128)

        self.fc4 = nn.Linear(128,64)
        self.fc5 = nn.Linear(64,4)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(0.2)

        self.norm2 = nn.BatchNorm1d(512)
        self.norm3 = nn.BatchNorm1d(128)
        self.norm4 = nn.BatchNorm1d(64)


    def forward(self, x):
        x = x.view(x.shape[0], -1)
        x = F.tanh(self.fc1(x))
        x = self.dropout(x)
        x = F.tanh(self.fc4(x))

        x = self.dropout(x)
        x = self.fc5(x) 

        return x

model = cadod_r()
# MSE loss scaffolding layer
loss_fn = torch.nn.MSELoss() 
optimizer = torch.optim.Adam(model.parameters(), lr=0.0005,weight_decay = 3e-4)

49152

loss_stats = {
    'train': [],
    "val": []
}

loss_fn = torch.nn.MSELoss(size_average=True)

optimizer = optim.Adam(model.parameters(), lr=0.0001, weight_decay = 0.005)

epochs = range(25)
count = 0 
running_loss = 0.0
for epoch in epochs:
    running_loss = 0.0
    for batch, data in enumerate(train_loader):
        # inputs, target = data[0].to(device), data[1].to(device)
        inputs, target = data[0], data[1]

        # Clear gradient buffers because we don't want any gradient from previous epoch to carry forward, dont want to cummulate gradients
        optimizer.zero_grad()

        # do forward pass
        output = model(inputs.float())
        #print(output.shape, target.shape)
        # compute loss and gradients
        loss = loss_fn(output, torch.unsqueeze(target.float(), dim=1))
        
        # get gradients w.r.t to parameters
        loss.backward()

        # perform gradient update
        optimizer.step()

        # print statistics
        running_loss += loss.item()*inputs.size(0)
        count += inputs.size(0)
    print(f"Epoch {epoch+1}, Test MSE loss: {np.round(running_loss/count, 3)}")
    loss_stats['train'].append(running_loss/count)

#print('Finished Training')
#Model evaluation on validation set
    count = 0 
    running_loss = 0.0
    test_size = 0
    model.eval()
    for batch, data in enumerate(valid_loader):
       
        inputs, target = data[0], data[1]
        # do forward pass
        output = model(inputs.float())
        # compute loss and gradients
        loss = loss_fn(output, torch.unsqueeze(target.float(), dim=1))
        running_loss += loss.item()*inputs.size(0)
        count += inputs.size(0) 
        test_size += batch_size

    print(f"Validation  MSE loss: {np.round(running_loss/count, 3)}")
    
    loss_stats['val'].append(running_loss/count)

print("Finished Training!")

count = 0 
running_loss = 0.0
test_size = 0
model.eval()
for batch, data in enumerate(test_loader):
    
    inputs, target = data[0], data[1]
    # do forward pass
    output = model(inputs.float())
    # compute loss and gradients
    loss = loss_fn(output, torch.unsqueeze(target.float(), dim=1))
    # print statistics
    running_loss += loss.item()*inputs.size(0)
    count += inputs.size(0) 
    test_size += batch_size
print(f" Test  MSE loss: {np.round(running_loss/count, 3)}")

# predict test
# model.to(device)

/usr/local/lib/python3.7/site-packages/torch/nn/functional.py:1794: UserWarning: nn.functional.tanh is deprecated. Use torch.tanh instead.
  warnings.warn("nn.functional.tanh is deprecated. Use torch.tanh instead.")
/usr/local/lib/python3.7/site-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([96, 1, 4])) that is different to the input size (torch.Size([96, 4])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([55, 1, 4])) that is different to the input size (torch.Size([55, 4])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)

Epoch 1, Test MSE loss: 0.495

/usr/local/lib/python3.7/site-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([1654, 1, 4])) that is different to the input size (torch.Size([1654, 4])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)

Validation  MSE loss: 0.418
Epoch 2, Test MSE loss: 0.325
Validation  MSE loss: 0.365
Epoch 3, Test MSE loss: 0.273
Validation  MSE loss: 0.31
Epoch 4, Test MSE loss: 0.224
Validation  MSE loss: 0.259
Epoch 5, Test MSE loss: 0.179
Validation  MSE loss: 0.208
Epoch 6, Test MSE loss: 0.133
Validation  MSE loss: 0.162
Epoch 7, Test MSE loss: 0.095
Validation  MSE loss: 0.128
Epoch 8, Test MSE loss: 0.071
Validation  MSE loss: 0.106
Epoch 9, Test MSE loss: 0.056
Validation  MSE loss: 0.087
Epoch 10, Test MSE loss: 0.043
Validation  MSE loss: 0.074
Epoch 11, Test MSE loss: 0.034
Validation  MSE loss: 0.062
Epoch 12, Test MSE loss: 0.028
Validation  MSE loss: 0.053
Epoch 13, Test MSE loss: 0.025
Validation  MSE loss: 0.045
Epoch 14, Test MSE loss: 0.02
Validation  MSE loss: 0.037
Epoch 15, Test MSE loss: 0.017
Validation  MSE loss: 0.034
Epoch 16, Test MSE loss: 0.015
Validation  MSE loss: 0.029
Epoch 17, Test MSE loss: 0.014
Validation  MSE loss: 0.025
Epoch 18, Test MSE loss: 0.013
Validation  MSE loss: 0.022
Epoch 19, Test MSE loss: 0.012
Validation  MSE loss: 0.02
Epoch 20, Test MSE loss: 0.011
Validation  MSE loss: 0.019
Epoch 21, Test MSE loss: 0.011
Validation  MSE loss: 0.016
Epoch 22, Test MSE loss: 0.011
Validation  MSE loss: 0.015
Epoch 23, Test MSE loss: 0.01
Validation  MSE loss: 0.014
Epoch 24, Test MSE loss: 0.01
Validation  MSE loss: 0.013
Epoch 25, Test MSE loss: 0.01
Validation  MSE loss: 0.012
Finished Training!
 Test  MSE loss: 0.013

/usr/local/lib/python3.7/site-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([1945, 1, 4])) that is different to the input size (torch.Size([1945, 4])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)

#Plotting train and avlidation MSE
import seaborn as sns
import matplotlib.pyplot as plt
train_val_loss_df = pd.DataFrame.from_dict(loss_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})

#fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(30,10))

sns.lineplot(data=train_val_loss_df, x = "epochs", y="value", hue="variable").set_title('Loss/Epoch')
plt.xlim([0,25])
plt.show()

expLog_RO = pd.DataFrame(columns=["exp_name", 
                               "Train MSE", 
                               "Valid MSE",
                               "Test  MSE",
                              ])

#Logging the experiment
exp_name = f"OOP API Regression"
expLog_RO.loc[0,:4] = [f"{exp_name}"] + list(np.round(
               [loss_stats['train'][-1], 
                loss_stats['val'][-1],
                running_loss/count],3))
expLog_RO

/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py:719: FutureWarning: Slicing a positional slice with .loc is not supported, and will raise TypeError in a future version.  Use .loc with labels or .iloc with positions instead.
  indexer = self._get_setitem_indexer(key)

	exp_name	Train MSE	Valid MSE	Test MSE
0	OOP API Regression	0.01	0.012	0.013

Results and Discussion

• Regularization and dropout value of 0.1 worked well for regression as well. Our best model yielded a close to zero MSE for training, validation and test data set

Multiheaded CXE + MSE Network

• Steps to create the model:

• Defined a model for classification and regression separately

• For each epoch, we calculated CXE and MSE loss from the above models and then used the combined loss for back propagation

• Adam optimizer was used to take the next step and update the weights and bias

import pandas as pd
import numpy as np
from sklearn.metrics import classification_report, confusion_matrix
from torch.optim import Adam
import matplotlib.pyplot as plt
import seaborn as sns

# Setting seeds to try and ensure we have the same results - this is not guaranteed across PyTorch releases.
import torch
torch.manual_seed(0)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False

import torch.nn as nn
import torch.nn.functional as F
from torchvision import datasets, transforms, models
from torch.utils.data import DataLoader, SubsetRandomSampler
import random
from os import listdir
from shutil import copyfile

import tensorflow as tf 
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from os import makedirs

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
device

device(type='cpu')

from tensorflow.keras import models
from tensorflow.keras.applications import *
from tensorflow.keras import layers

Training from the MultiLayer Perceptron

#creating directories for test and train data set
dataset_home = 'cat_vs_dog/'
subdirs = ['train/', 'test/']
for subdir in subdirs:
	# create label subdirectories
	labeldirs = ['dogs/', 'cats/']
	for labldir in labeldirs:
		newdir = dataset_home + subdir + labldir
		makedirs(newdir, exist_ok=True)

#load the csv file
df = pd.read_csv("cadod.csv")
df.head()

	ImageID	Source	LabelName	Confidence	XMin	XMax	YMin	YMax	IsOccluded	IsTruncated	...	IsDepiction	IsInside	XClick1X	XClick2X	XClick3X	XClick4X	XClick1Y	XClick2Y	XClick3Y	XClick4Y
0	0000b9fcba019d36	xclick	/m/0bt9lr	1	0.165000	0.903750	0.268333	0.998333	1	1	...	0	0	0.636250	0.903750	0.748750	0.165000	0.268333	0.506667	0.998333	0.661667
1	0000cb13febe0138	xclick	/m/0bt9lr	1	0.000000	0.651875	0.000000	0.999062	1	1	...	0	0	0.312500	0.000000	0.317500	0.651875	0.000000	0.410882	0.999062	0.999062
2	0005a9520eb22c19	xclick	/m/0bt9lr	1	0.094167	0.611667	0.055626	0.998736	1	1	...	0	0	0.487500	0.611667	0.243333	0.094167	0.055626	0.226296	0.998736	0.305942
3	0006303f02219b07	xclick	/m/0bt9lr	1	0.000000	0.999219	0.000000	0.998824	1	1	...	0	0	0.508594	0.999219	0.000000	0.478906	0.000000	0.375294	0.720000	0.998824
4	00064d23bf997652	xclick	/m/0bt9lr	1	0.240938	0.906183	0.000000	0.694286	0	0	...	0	0	0.678038	0.906183	0.240938	0.522388	0.000000	0.370000	0.424286	0.694286

5 rows × 21 columns

df.LabelName.replace({'/m/01yrx':'cat', '/m/0bt9lr':'dog'}, inplace=True)

dog_list = df[df.LabelName == 'dog']['ImageID']
cat_list = df[df.LabelName == 'cat']['ImageID']
#list(dog_list)

file_name_test = []
file_name_train = []

# moving images to test and train folder

random.seed(10)
# define ratio of pictures to use for test
test_ratio = 0.20
count_c = 0
count_d = 0

# copy training dataset images into subdirectories
src_directory = 'cadod/'
for file in listdir(src_directory):
    #print(file.replace('.jpg','').replace('._','') in list(cat_list))
    #print(file.replace('.jpg','').replace('._','') in list(dog_list))
    src = src_directory + '/' + file
    dst_dir = 'train/'
    if random.random() < test_ratio:
        dst_dir = 'test/'
        file_name_test.append(file.replace('.jpg','').replace('._',''))
    if file.replace('.jpg','').replace('._','') in list(cat_list) and count_c < 500:
        dst = dataset_home + dst_dir + 'cats/'  + file
        count_c +=1
        copyfile(src, dst)
        file_name_train.append(file.replace('.jpg','').replace('._',''))
    elif file.replace('.jpg','').replace('._','') in list(dog_list) and count_d < 500:
        dst = dataset_home + dst_dir + 'dogs/'  + file
        count_d +=1
        copyfile(src, dst)
        file_name_train.append(file.replace('.jpg','').replace('._',''))

train_id = pd.DataFrame (file_name_train, columns = ['ImageID'])
train_id.head()

	ImageID
0	2b55a824f4a375d3
1	bc26925fd646efe5
2	f8fccbefa2e8e33f
3	9c730899f38007cc
4	01901b6370020f3c

test_id = pd.DataFrame (file_name_test, columns = ['ImageID'])
test_id.head()

	ImageID
0	d00eb685487904b0
1	ddfc5237d20952a7
2	e60d548f2f124a01
3	f4add7bb2ee11f8d
4	8054527db8754ab1

df.ImageID.astype('O')
train_id.ImageID.astype('O')
df_n = df[['ImageID','XMin', 'YMin', 'XMax', 'YMax']]
df_n.set_index('ImageID')
train_id.set_index('ImageID')
train_id_n = df_n.join(train_id, how = 'left', lsuffix = '_left', rsuffix = '_right')
train_id_n.drop(columns = ['ImageID_right'], inplace = True)
train_id_n.head(5)

	ImageID_left	XMin	YMin	XMax	YMax
0	0000b9fcba019d36	0.165000	0.268333	0.903750	0.998333
1	0000cb13febe0138	0.000000	0.000000	0.651875	0.999062
2	0005a9520eb22c19	0.094167	0.055626	0.611667	0.998736
3	0006303f02219b07	0.000000	0.000000	0.999219	0.998824
4	00064d23bf997652	0.240938	0.000000	0.906183	0.694286

df.ImageID.astype('O')
test_id.ImageID.astype('O')
#df.set_index('ImageID')
test_id.set_index('ImageID')
test_id_n = df_n.join(test_id, how = 'left', lsuffix = '_left', rsuffix = '_right')
test_id_n.drop(columns = ['ImageID_right'], inplace = True)
test_id_n.head(5)

	ImageID_left	XMin	YMin	XMax	YMax
0	0000b9fcba019d36	0.165000	0.268333	0.903750	0.998333
1	0000cb13febe0138	0.000000	0.000000	0.651875	0.999062
2	0005a9520eb22c19	0.094167	0.055626	0.611667	0.998736
3	0006303f02219b07	0.000000	0.000000	0.999219	0.998824
4	00064d23bf997652	0.240938	0.000000	0.906183	0.694286

expLog = pd.DataFrame(columns=["exp_name", 
                               "Train Loss", 
                               "Valid Loss",
                               "Test Loss",
                              ])

Image Classification

mean = [0.485, 0.456, 0.406]
std = [0.229, 0.224, 0.225]

transform_test = transforms.Compose([
    #transforms.ToPILImage(),             
    transforms.Resize((128, 128)),
    transforms.ToTensor(),
    transforms.Normalize(mean=mean, std=std)
    ])
transform_train = transforms.Compose([
    #transforms.ToPILImage(),             
    transforms.Resize((128, 128)),
    transforms.RandomHorizontalFlip(),
    transforms.RandomRotation(40),
    transforms.ToTensor(),
    transforms.Normalize(mean=mean, std=std)
    #transforms.RandomAutocontrast()
    
    ])

train_it = datasets.ImageFolder('cat_vs_dog/train/', transform=transform_train)
test_it = datasets.ImageFolder('cat_vs_dog/test/', transform=transform_test)

dataset_size = len(train_it)
dataset_indices = list(range(dataset_size))
np.random.shuffle(dataset_indices)

dataset_size

800

idx2class = {v: k for k, v in train_it.class_to_idx.items()}
idx2class

{0: 'cats', 1: 'dogs'}

dataset_size = len(train_it)
dataset_indices = list(range(dataset_size))
#dataset_indices[val_split_index:]

np.random.shuffle(dataset_indices)

val_split_index = int(np.floor(0.2 * dataset_size))

train_idx, val_idx = dataset_indices[val_split_index:], dataset_indices[:val_split_index]

train_sampler = SubsetRandomSampler(train_idx)
val_sampler = SubsetRandomSampler(val_idx)

bs_train = 16
bs_test = 4
bs_valid = 8
trainloader = DataLoader(dataset=train_it, shuffle=False, batch_size=bs_train, sampler=train_idx)
valloader = DataLoader(dataset=train_it, shuffle=False, batch_size=bs_valid, sampler=val_idx)
testloader = DataLoader(test_it, batch_size=bs_test, shuffle=False)

y_box_train = train_id_n[val_split_index:]
y_box_val =  train_id_n[:val_split_index]
y_box_val.shape

(160, 5)

for images, labels in trainloader:
    print(images.size(), labels.size())
    print(labels)
    break

torch.Size([16, 3, 128, 128]) torch.Size([16])
tensor([1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0])

for images, labels in valloader:
    print(images.size(), labels.size())
    print(labels)
    break

torch.Size([8, 3, 128, 128]) torch.Size([8])
tensor([0, 0, 1, 1, 0, 1, 0, 1])

import numpy as np
X = np.load('data/img.npy', allow_pickle=True)
y_label = np.load('data/y_label.npy', allow_pickle=True)
y_bbox = np.load('data/y_bbox.npy', allow_pickle=True)

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler

y_train_tensor = torch.from_numpy(y_box_train[['XMin', 'YMin', 'XMax', 'YMax']].to_numpy())
y_val_tensor = torch.from_numpy(y_box_val[['XMin', 'YMin', 'XMax', 'YMax']].to_numpy())
y_test_tensor = torch.from_numpy(test_id_n[['XMin', 'YMin', 'XMax', 'YMax']].to_numpy())
#y_val_tensor

Classifier model

from torch.optim import Adam
class cadod_c(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(128*128*3, 128)
        #self.fc2 = nn.Linear(1024,512)
        #self.fc3 = nn.Linear(512,128)
        self.fc4 = nn.Linear(128,64)
        self.fc5 = nn.Linear(64,2)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(0.1)
        #self.norm1 = nn.BatchNorm1d(1024)
        self.norm2 = nn.BatchNorm1d(512)
        self.norm3 = nn.BatchNorm1d(128)
        self.norm4 = nn.BatchNorm1d(64)


    def forward(self, x):
        x = x.view(x.shape[0], -1)
        x = F.relu(self.fc1(x))

        x = self.norm3(x)
        x = self.dropout(x)
        x = F.relu(self.fc4(x))
        x = self.norm4(x)
        x =self.dropout(x)
        x = self.fc5(x) 
        x = F.softmax(x, dim = 1)
        return x

model_c = cadod_c()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model_c.parameters(), lr=0.0002, weight_decay = 3e-3)
scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones = [500,1000,1500], gamma = 0.5)

Regressor model

D_in = 128*128*3
print(D_in)
D_hidden = 128
D_out = 4
# Use the nn package to define our model and loss function.
# use the sequential API makes things simple
class cadod_r(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(D_in, 128)
        #self.fc2 = nn.Linear(1024,512)
        #self.fc3 = nn.Linear(512,128)
        self.fc4 = nn.Linear(128,64)
        self.fc5 = nn.Linear(64,4)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(0.2)
        #self.norm1 = nn.BatchNorm1d(1024)
        self.norm2 = nn.BatchNorm1d(512)
        self.norm3 = nn.BatchNorm1d(128)
        self.norm4 = nn.BatchNorm1d(64)


    def forward(self, x):
        x = x.view(x.shape[0], -1)
        x = F.tanh(self.fc1(x))
        #x = self.norm1(x)
        #x = F.relu(self.fc2(x))
        #x = self.norm2(x)
        #x = F.relu(self.fc3(x))
        #x = self.norm3(x)
        x = self.dropout(x)
        x = F.tanh(self.fc4(x))
        #x = self.norm4(x)
        x = self.dropout(x)
        x = self.fc5(x) 
        #x = F.softmax(x, dim = 1)
        return x

model_r = cadod_r()
# MSE loss scaffolding layer
loss_fn = torch.nn.MSELoss() 
optimizer = torch.optim.Adam(model_r.parameters(), lr=0.0005,weight_decay = 3e-4)

49152

accuracy_stats = {
    'train': [],
    "val": []
}
loss_stats = {
    'train': [],
    "val": []
}

def binary_acc(y_pred, y_test):
    y_pred_tag = torch.log_softmax(y_pred, dim = 1)
    _, y_pred_tags = torch.max(y_pred_tag, dim = 1)    
    correct_results_sum = (y_pred_tags == y_test).sum().float()    
    acc = correct_results_sum/y_test.shape[0]
    acc = torch.round(acc * 100)    
    return acc

Multi headed model

model_c = model_c.to(device)
model_r = model_r.to(device)


num_epochs = 50


for e in range(num_epochs):
    cum_epoch_loss = 0
    cum_acc = 0
    batch_loss = 0
    mse_loss = 0
    train = 0
    test = 0
    val = 0
    model_c.train()
    model_r.train()
    #Training the model
    for batch, (images, labels) in enumerate(trainloader,1):
        
        images = images.to(device)
        labels = labels.to(device)
        #images_2 = images_2.to(device)
        
        bbox = y_train_tensor[train:train+bs_train].to(device)
        train +=bs_train
        # Clear gradient buffers because we don't want any gradient from previous epoch to carry forward, dont want to cummulate gradients
        optimizer.zero_grad()
        label_pred = model_c(images).squeeze()  #training the classifier model
        box_pred = model_r(images)  #training the regressor model
        loss_1 = criterion(label_pred, labels) #CXE loss
        acc = binary_acc(label_pred, labels)
        loss_2 = loss_fn(box_pred, torch.unsqueeze(bbox.float(), dim=1)) #MSE
        loss = loss_1 + loss_2 #combined loss
        loss.backward() #backpropagating loss
        optimizer.step()  #gradient update
        batch_loss += loss.item()
        cum_acc += acc.item()
        
        scheduler.step()
        #print(f'Epoch({e}/{num_epochs} : Batch number({batch}/{len(trainloader)})')
    
    #Evaluating the model on validation set
    with torch.no_grad():
        model_c.eval()
        model_r.eval()
        val_epoch_loss = 0
        val_epoch_acc = 0
        
        val = 0
        for batch, (X_val_batch, y_val_batch) in enumerate(valloader,1):
            X_val_batch, y_val_batch = X_val_batch.to(device), y_val_batch.to(device)
            y_box_val = y_val_tensor[val:val+bs_valid].to(device)
            
            y_val_pred = model_c(X_val_batch).squeeze()
            y_box_pred = model_r(X_val_batch).squeeze()
                        
            val_loss = criterion(y_val_pred, y_val_batch)
            val_acc = binary_acc(y_val_pred, y_val_batch) 
            
            mse_loss = loss_fn(y_box_pred, torch.unsqueeze(y_box_val.float(), dim=1))
            
            val_epoch_loss += val_loss.item() + mse_loss.item()
            val_epoch_acc += val_acc.item()
            val += bs_valid
            
     #saving the results for plotting   
    loss_stats['train'].append(batch_loss/len(trainloader))
    loss_stats['val'].append(val_epoch_loss/len(valloader))
    accuracy_stats['train'].append(cum_acc/len(trainloader))
    accuracy_stats['val'].append(val_epoch_acc/len(valloader))
    
    print(f'Epoch({e}/{num_epochs})')
    print(f'Training loss : {batch_loss/len(trainloader)}')  
    print(f'Training accuracy : {cum_acc/len(trainloader)}')  
    print(f'Validation loss : {val_epoch_loss/len(valloader)}')  
    print(f'Validation accuracy : {val_epoch_acc/len(valloader)}') 
    

/usr/local/lib/python3.7/site-packages/torch/nn/functional.py:1794: UserWarning: nn.functional.tanh is deprecated. Use torch.tanh instead.
  warnings.warn("nn.functional.tanh is deprecated. Use torch.tanh instead.")
/usr/local/lib/python3.7/site-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([16, 1, 4])) that is different to the input size (torch.Size([16, 4])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)
/usr/local/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:134: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([8, 1, 4])) that is different to the input size (torch.Size([8, 4])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)

Epoch(0/50)
Training loss : 0.879992139339447
Training accuracy : 47.05
Validation loss : 0.8160292257554829
Validation accuracy : 42.0
Epoch(1/50)
Training loss : 0.8147073850035668
Training accuracy : 50.225
Validation loss : 0.7857627982273698
Validation accuracy : 43.8
Epoch(2/50)
Training loss : 0.797081308066845
Training accuracy : 47.95
Validation loss : 0.7427065202966332
Validation accuracy : 56.65
Epoch(3/50)
Training loss : 0.7843752071261406
Training accuracy : 46.55
Validation loss : 0.7327043810859323
Validation accuracy : 54.35
Epoch(4/50)
Training loss : 0.776068665087223
Training accuracy : 47.15
Validation loss : 0.7662655789870769
Validation accuracy : 44.55
Epoch(5/50)
Training loss : 0.778938265144825
Training accuracy : 48.275
Validation loss : 0.7574834938161075
Validation accuracy : 49.35
Epoch(6/50)
Training loss : 0.7841269075870514
Training accuracy : 49.75
Validation loss : 0.7543694017920644
Validation accuracy : 47.45
Epoch(7/50)
Training loss : 0.7646772757172584
Training accuracy : 47.6
Validation loss : 0.7452828153967858
Validation accuracy : 45.65
Epoch(8/50)
Training loss : 0.7659271940588951
Training accuracy : 47.875
Validation loss : 0.7139661338180303
Validation accuracy : 51.9
Epoch(9/50)
Training loss : 0.7437738716602326
Training accuracy : 53.95
Validation loss : 0.7309343723580242
Validation accuracy : 53.7
Epoch(10/50)
Training loss : 0.7507725208997726
Training accuracy : 49.225
Validation loss : 0.7389657015912234
Validation accuracy : 48.25
Epoch(11/50)
Training loss : 0.7487147629261017
Training accuracy : 48.925
Validation loss : 0.7363098844420165
Validation accuracy : 47.6
Epoch(12/50)
Training loss : 0.7588467597961426
Training accuracy : 47.975
Validation loss : 0.7368810486514121
Validation accuracy : 49.95
Epoch(13/50)
Training loss : 0.7440685167908668
Training accuracy : 48.45
Validation loss : 0.7334678072016686
Validation accuracy : 46.3
Epoch(14/50)
Training loss : 0.7444633960723877
Training accuracy : 48.65
Validation loss : 0.7298034369014204
Validation accuracy : 45.0
Epoch(15/50)
Training loss : 0.740554977953434
Training accuracy : 47.9
Validation loss : 0.7358317765872926
Validation accuracy : 45.05
Epoch(16/50)
Training loss : 0.7501006409525871
Training accuracy : 48.55
Validation loss : 0.7311018154490739
Validation accuracy : 44.4
Epoch(17/50)
Training loss : 0.7362614899873734
Training accuracy : 50.65
Validation loss : 0.7324363643303513
Validation accuracy : 46.25
Epoch(18/50)
Training loss : 0.7371111124753952
Training accuracy : 48.725
Validation loss : 0.7222464400343597
Validation accuracy : 46.95
Epoch(19/50)
Training loss : 0.7421576544642449
Training accuracy : 51.4
Validation loss : 0.7395074652973562
Validation accuracy : 45.6
Epoch(20/50)
Training loss : 0.7350664213299751
Training accuracy : 50.475
Validation loss : 0.7208672075299546
Validation accuracy : 51.9
Epoch(21/50)
Training loss : 0.7439446061849594
Training accuracy : 46.85
Validation loss : 0.7228649198077619
Validation accuracy : 51.9
Epoch(22/50)
Training loss : 0.7273421421647072
Training accuracy : 51.175
Validation loss : 0.7153717894572764
Validation accuracy : 47.45
Epoch(23/50)
Training loss : 0.7386784791946411
Training accuracy : 47.175
Validation loss : 0.7181700155604631
Validation accuracy : 50.7
Epoch(24/50)
Training loss : 0.7280746743083
Training accuracy : 50.675
Validation loss : 0.7240190807729959
Validation accuracy : 44.5
Epoch(25/50)
Training loss : 0.7131540060043335
Training accuracy : 53.3
Validation loss : 0.7357481085695327
Validation accuracy : 44.45
Epoch(26/50)
Training loss : 0.7224125787615776
Training accuracy : 48.3
Validation loss : 0.7455177581869066
Validation accuracy : 43.75
Epoch(27/50)
Training loss : 0.7358808040618896
Training accuracy : 48.425
Validation loss : 0.7191734137479215
Validation accuracy : 48.25
Epoch(28/50)
Training loss : 0.7321872726082802
Training accuracy : 49.45
Validation loss : 0.7157092281617224
Validation accuracy : 50.6
Epoch(29/50)
Training loss : 0.7283739775419236
Training accuracy : 49.225
Validation loss : 0.7252140298951417
Validation accuracy : 46.85
Epoch(30/50)
Training loss : 0.7236504539847374
Training accuracy : 50.725
Validation loss : 0.7287843666737899
Validation accuracy : 46.15
Epoch(31/50)
Training loss : 0.7320128247141838
Training accuracy : 47.85
Validation loss : 0.721864893520251
Validation accuracy : 44.95
Epoch(32/50)
Training loss : 0.7290561392903327
Training accuracy : 48.475
Validation loss : 0.7366988216992467
Validation accuracy : 43.9
Epoch(33/50)
Training loss : 0.7262326002120971
Training accuracy : 50.65
Validation loss : 0.7219362199073658
Validation accuracy : 43.8
Epoch(34/50)
Training loss : 0.718777984380722
Training accuracy : 52.6
Validation loss : 0.727155020320788
Validation accuracy : 45.6
Epoch(35/50)
Training loss : 0.737250167131424
Training accuracy : 46.3
Validation loss : 0.7392244618851691
Validation accuracy : 43.25
Epoch(36/50)
Training loss : 0.7214906260371208
Training accuracy : 49.725
Validation loss : 0.7153489050921052
Validation accuracy : 50.6
Epoch(37/50)
Training loss : 0.7369486078619957
Training accuracy : 48.3
Validation loss : 0.7168718978296965
Validation accuracy : 51.25
Epoch(38/50)
Training loss : 0.7245754212141037
Training accuracy : 53.425
Validation loss : 0.721158375730738
Validation accuracy : 47.55
Epoch(39/50)
Training loss : 0.7302716836333275
Training accuracy : 49.275
Validation loss : 0.7130527328234166
Validation accuracy : 48.85
Epoch(40/50)
Training loss : 0.7254614874720573
Training accuracy : 50.4
Validation loss : 0.7129096592310816
Validation accuracy : 51.3
Epoch(41/50)
Training loss : 0.7377575367689133
Training accuracy : 46.625
Validation loss : 0.7053902443498373
Validation accuracy : 51.15
Epoch(42/50)
Training loss : 0.7279296696186066
Training accuracy : 48.65
Validation loss : 0.730576116265729
Validation accuracy : 41.95
Epoch(43/50)
Training loss : 0.7306485831737518
Training accuracy : 47.825
Validation loss : 0.7385534157976508
Validation accuracy : 42.05
Epoch(44/50)
Training loss : 0.7235222369432449
Training accuracy : 48.725
Validation loss : 0.7107356958091259
Validation accuracy : 48.8
Epoch(45/50)
Training loss : 0.7196284979581833
Training accuracy : 51.65
Validation loss : 0.7136089261621237
Validation accuracy : 48.35
Epoch(46/50)
Training loss : 0.7213528364896774
Training accuracy : 50.25
Validation loss : 0.7174238551873714
Validation accuracy : 51.75
Epoch(47/50)
Training loss : 0.7155992642045022
Training accuracy : 51.4
Validation loss : 0.7445846081245691
Validation accuracy : 44.4
Epoch(48/50)
Training loss : 0.7200534373521805
Training accuracy : 50.975
Validation loss : 0.7272249827627093
Validation accuracy : 46.25
Epoch(49/50)
Training loss : 0.7285032212734223
Training accuracy : 48.25
Validation loss : 0.7168691314291209
Validation accuracy : 54.2

#Plotting training and validation loss

train_val_loss_df = pd.DataFrame.from_dict(loss_stats).reset_index().melt(id_vars=['index']).rename(columns={"index":"epochs"})

sns.lineplot(data=train_val_loss_df, x = "epochs", y="value", hue="variable").set_title('Loss/Epoch')

plt.show()

#Getting testing loss
y_pred_list = []
y_true_list = []
#model.eval()
#with torch.no_grad():
with torch.no_grad():
        model_c.eval()
        model_r.eval()
        test_epoch_loss = 0
        test_epoch_acc = 0
        #val_mse_loss = 0
        test = 0
        for batch, (X_test_batch, y_test_batch) in enumerate(testloader,1):
            X_test_batch, y_test_batch = X_test_batch.to(device), y_test_batch.to(device)
            y_box_test = y_test_tensor[test:test+bs_valid].to(device)
            #print(y_box_val)
            y_test_pred = model_c(X_test_batch).squeeze()
            y_box_pred = model_r(X_test_batch).squeeze()
            #y_val_pred = torch.unsqueeze(y_val_pred, 0)            
            test_loss = criterion(y_test_pred, y_test_batch)
            test_acc = binary_acc(y_test_pred, y_test_batch) 
            #print(y_box_val)
            #print(y_box_pred)
            mse_loss = loss_fn(y_box_pred, torch.unsqueeze(y_box_test.float(), dim=1))
            #print(val_loss.item(),mse_loss.item())
            test_epoch_loss += test_loss.item() + mse_loss.item()
            test_epoch_acc += test_acc.item()
            test += bs_valid
            
        

        #print(f'Epoch({e}/{num_epochs})')
        print(f'Test loss : {test_epoch_loss/len(testloader)}')  
          
        

/usr/local/lib/python3.7/site-packages/torch/nn/functional.py:1794: UserWarning: nn.functional.tanh is deprecated. Use torch.tanh instead.
  warnings.warn("nn.functional.tanh is deprecated. Use torch.tanh instead.")
/usr/local/lib/python3.7/site-packages/torch/nn/modules/loss.py:528: UserWarning: Using a target size (torch.Size([8, 1, 4])) that is different to the input size (torch.Size([4, 4])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)

Test loss : 0.7493907929956913

Result and Discussions

• Combining the loss of classification and regression model to backpropagate the loss to improve the model

• We trained the model for 50 epochs and saw that both train and validation loss decreased steadily after each epoch

  • At the end of all the epochs, we ended up with a loss close to 0.7

Conclusion:

Our main objective was to classify images of cats and dogs and to identify the location. This fundamental problem in computer vision is the basis of many other computer vision tasks. In the phase, we used neural networks with data augmentation, drop out layers and regularization to make the prediction, which reduced overfitting the model and yielded us with an accuracy of ~60%. In addition to this, we also combined the CXE and MSE loss and then sent it back for optimization of the neural network which helped us reduce the loss steadily after each epoch.

Our next steps will be as follows:

• Fine tune the existing CNN model by adding dropout layers, batchnorm layer, and maxpooling layer
  • Also, we plan to experiment with regularization and different types of optimizers
• Build a transfer learning network using efficient det
• Perform a t-test to compare the baseline model with the improved model to make sure our improved model performs significantly better than the models from the previous phases